mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-20 06:51:59 +00:00
🗂️ feat: Audit MCP, Code Worker, Langfuse, Skill Credential, Passkey and Audit Log Access (#17054)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/chat (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/chat (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
* fix: Remove Group Dependents Before Deleting the Group * feat: Audit Code Worker Pairing and Langfuse Connection Changes * feat: Audit Skill Sync Credential Changes * feat: Audit Passkey Enrolment and Removal * feat: Audit Audit Log Reads and CSV Exports * feat: Audit Audit Log Retention Purges * feat: Audit MCP Server Create, Update and Delete * fix: Keep Purge Result When the Purge Audit Write Fails * test: Cover Admin Integration Audit Scenarios * fix: Tighten Admin Integration Audit Scope and Group Cleanup Order * fix: Run Security Fences Before Awaiting Audit Writes * fix: Move MCP Post-Commit Audit Ordering Into the Auditor and Audit Verify Reads * fix: Fail Closed When Recording Audit Log Access * fix: Audit Group Deletion Before the Cascade and Restore Dependents on Failure * fix: Restore Only Config Overrides When Group Deletion Fails * fix: Audit Worker Pairing and Export Before the Work and Append Failures * fix: Record Group Deletion as Pending Without Delaying Revocation * fix: Skip Config Restore When a Concurrent Delete Removed the Group * fix: Restore Group Config When the Existence Recheck Fails * fix: Keep Group Config Restricted When Delete State Is Unknown * fix: Record Worker Pairing and Revocation as Pending Until Confirmed * fix: Deliver the Pairing Code Before the Terminal Audit Write * test: Assert Pairing Code Delivery Precedes the Terminal Audit Write * fix: Send Revocation and Group Deletion Responses Before the Terminal Audit Write * fix: Snapshot Group Config Before the Destructive Cascade Step * refactor: Type the Group Config Snapshot Dependency Without Mongoose Shapes * fix: Fence Shared MCP Users Before the Deletion Audit and Correlate Audit Attempts * fix: Diff MCP Server Audits Against the Committed Config * fix: Read the Committed MCP Config at Commit Time for the Audit Diff * refactor: Start the Committed MCP Config Read Inside the Auditor * fix: Audit Langfuse Patches When Activation Fails and Surface Failed Group Config Restores * fix: Record Member Removals When Group Deletion Completes After a Failed Restore * test: Serve the Committed Config on the MCP Update Audit Reread * fix: Audit the Group Returned by the Compensating Deletion * fix: Keep Member Removal Audits Fail-Open During Group Deletion * fix: Retry ACL Cleanup After a Compensating Group Deletion * fix: Mark Unavailable Audit Detail and Order Member Removals After the Pending Deletion * fix: Guard Synchronous Committed-Config Read Failures in the MCP Auditor * fix: Return the Persisted MCP Config and Audit Commits That Outlive Cache Invalidation * style: Sort Imports in MCP Audit Files * style: Sort Imports After Restack * fix: Audit MCP Commits That Outlive Any Post-Write Step and Pend Audit Exports Until Complete * test: Provide the Fail-Closed Setting in the Code Route Mock * fix: Refresh MCP Read Caches When a Repository Step Fails After the Write * fix: Finish Audit Exports on Response Completion and Keep Error Details Out of Messages and Logs * fix: Watch Response Completion Before the Truncation Footer Write * fix: Skip Audit for No-op MCP Deletes and Close Remaining Post-Failure Audit Gaps * fix: Keep Causes Out of Structured Logs and Treat a Lost Compensating Delete as Not Found * fix: Mark Failed MCP Post-Commit Steps and Null Langfuse Activation in Audits
This commit is contained in:
parent
76192bdfd0
commit
9adabde72e
36 changed files with 2758 additions and 147 deletions
|
|
@ -20,4 +20,5 @@ module.exports = createPasskeyHandlers({
|
|||
findPasskeysByUser: db.findPasskeysByUser,
|
||||
countPasskeysByUser: db.countPasskeysByUser,
|
||||
findPasskeyByCredentialId: db.findPasskeyByCredentialId,
|
||||
recordAuditEntry: db.recordAuditEntry,
|
||||
});
|
||||
|
|
|
|||
|
|
@ -20,6 +20,8 @@ const {
|
|||
redactServerSecrets,
|
||||
sanitizeMcpIconPath,
|
||||
redactAllServerSecrets,
|
||||
createMCPServerAuditor,
|
||||
settleMCPCommit,
|
||||
getMCPErrorResponse,
|
||||
prepareMCPServerOAuthDeletion,
|
||||
cleanupDeletedMCPServerOAuthUsers,
|
||||
|
|
@ -52,6 +54,8 @@ const { hasCapability } = require('~/server/middleware/roles/capabilities');
|
|||
const { getMCPManager, getMCPServersRegistry } = require('~/config');
|
||||
const db = require('~/models');
|
||||
|
||||
const auditMCPServer = createMCPServerAuditor({ recordAuditEntry: db.recordAuditEntry });
|
||||
|
||||
/**
|
||||
* Handles MCP-specific errors and sends appropriate HTTP responses.
|
||||
* @param {Error} error - The error to handle
|
||||
|
|
@ -457,18 +461,30 @@ const createMCPServerController = async (req, res) => {
|
|||
...new Set([...configNames, ...configNames.map(normalizeServerName)]),
|
||||
];
|
||||
const inspectionServerName = `temp_server_${randomUUID()}`;
|
||||
let result;
|
||||
let settled;
|
||||
try {
|
||||
result = await getMCPServersRegistry().addServer(
|
||||
inspectionServerName,
|
||||
validation.data,
|
||||
'DB',
|
||||
userId,
|
||||
reservedServerNames,
|
||||
settled = await settleMCPCommit(() =>
|
||||
getMCPServersRegistry().addServer(
|
||||
inspectionServerName,
|
||||
validation.data,
|
||||
'DB',
|
||||
userId,
|
||||
reservedServerNames,
|
||||
),
|
||||
);
|
||||
} finally {
|
||||
MCPConnection.clearCooldown(inspectionServerName);
|
||||
}
|
||||
const result = settled.committed;
|
||||
await auditMCPServer(req, {
|
||||
action: 'mcp_server.created',
|
||||
serverName: result.serverName,
|
||||
next: result.config,
|
||||
postWriteFailed: settled.postWriteError != null,
|
||||
});
|
||||
if (settled.postWriteError) {
|
||||
throw settled.postWriteError;
|
||||
}
|
||||
res.status(201).json({
|
||||
serverName: result.serverName,
|
||||
...redactServerSecrets(result.config, { canEdit: true }),
|
||||
|
|
@ -572,8 +588,11 @@ const updateMCPServerController = async (req, res) => {
|
|||
);
|
||||
const retainedTools = await getMCPServerTools(userId, serverName, existingConfig);
|
||||
await invalidateCachedTools({ userId, serverName });
|
||||
let settled;
|
||||
try {
|
||||
await registry.commitServerUpdate(serverName, parsedConfig, 'DB', userId);
|
||||
settled = await settleMCPCommit(() =>
|
||||
registry.commitServerUpdate(serverName, parsedConfig, 'DB', userId),
|
||||
);
|
||||
} catch (error) {
|
||||
await restoreRetainedServerCatalog({
|
||||
userId,
|
||||
|
|
@ -583,12 +602,28 @@ const updateMCPServerController = async (req, res) => {
|
|||
});
|
||||
throw error;
|
||||
}
|
||||
const committedConfig = settled.committed ?? parsedConfig;
|
||||
/** Fence connections another replica could have created from the old DB
|
||||
* config between the pre-commit fence and the committed update. */
|
||||
await fenceCommittedMCPMutation({ userId, serverName });
|
||||
await disconnectLocalMCPServer(userId, serverName);
|
||||
await auditMCPServer(
|
||||
req,
|
||||
{
|
||||
action: 'mcp_server.updated',
|
||||
serverName,
|
||||
previous: existingConfig,
|
||||
next: committedConfig,
|
||||
postWriteFailed: settled.postWriteError != null,
|
||||
},
|
||||
async () => {
|
||||
await fenceCommittedMCPMutation({ userId, serverName });
|
||||
await disconnectLocalMCPServer(userId, serverName);
|
||||
},
|
||||
);
|
||||
if (settled.postWriteError) {
|
||||
throw settled.postWriteError;
|
||||
}
|
||||
|
||||
res.status(200).json(redactServerSecrets(parsedConfig, { canEdit: true }));
|
||||
res.status(200).json(redactServerSecrets(committedConfig, { canEdit: true }));
|
||||
} catch (error) {
|
||||
logger.error('[updateMCPServer]', error);
|
||||
const mcpErrorResponse = handleMCPError(error, res);
|
||||
|
|
@ -631,8 +666,9 @@ const deleteMCPServerController = async (req, res, uninstallOAuthMCP) => {
|
|||
getMCPServerTools(userId, serverName, existingConfig),
|
||||
]);
|
||||
await invalidateCachedTools({ userId, serverName });
|
||||
let settled;
|
||||
try {
|
||||
await registry.removeServer(serverName, 'DB', userId);
|
||||
settled = await settleMCPCommit(() => registry.removeServer(serverName, 'DB', userId));
|
||||
} catch (error) {
|
||||
await restoreRetainedServerCatalog({
|
||||
userId,
|
||||
|
|
@ -642,34 +678,51 @@ const deleteMCPServerController = async (req, res, uninstallOAuthMCP) => {
|
|||
});
|
||||
throw error;
|
||||
}
|
||||
/** Fence connections another replica could have created before deletion committed. */
|
||||
await fenceCommittedMCPMutation({ userId, serverName });
|
||||
await disconnectLocalMCPServer(userId, serverName);
|
||||
try {
|
||||
await cleanupDeletedMCPServerOAuthUsers({
|
||||
ownerUserId: userId,
|
||||
/** Fence connections another replica could have created before deletion committed,
|
||||
* for every user holding a token, before the audit write is awaited. */
|
||||
await auditMCPServer(
|
||||
req,
|
||||
{
|
||||
action: 'mcp_server.deleted',
|
||||
serverName,
|
||||
serverConfig: existingConfig,
|
||||
snapshot: oauthDeletionSnapshot,
|
||||
getTokenUserIds,
|
||||
getUserPrincipals: (candidateUserId) => db.getUserPrincipals({ userId: candidateUserId }),
|
||||
resolveAllowlists: (candidateUserId) =>
|
||||
registry.resolveAllowlists({ userId: candidateUserId }),
|
||||
fenceAndDisconnectUser: async (candidateUserId) => {
|
||||
if (candidateUserId === userId) {
|
||||
return;
|
||||
}
|
||||
await fenceCommittedMCPMutation({ userId: candidateUserId, serverName });
|
||||
await disconnectLocalMCPServer(candidateUserId, serverName);
|
||||
},
|
||||
uninstallOAuthMCP,
|
||||
});
|
||||
} catch (error) {
|
||||
logger.warn(
|
||||
`[deleteMCPServer] Server ${serverName} was deleted, but OAuth cleanup failed for user ${userId}:`,
|
||||
error,
|
||||
);
|
||||
throw error;
|
||||
previous: existingConfig,
|
||||
postWriteFailed: settled.postWriteError != null,
|
||||
noop: settled.committed === false,
|
||||
},
|
||||
async () => {
|
||||
await fenceCommittedMCPMutation({ userId, serverName });
|
||||
await disconnectLocalMCPServer(userId, serverName);
|
||||
try {
|
||||
await cleanupDeletedMCPServerOAuthUsers({
|
||||
ownerUserId: userId,
|
||||
serverName,
|
||||
serverConfig: existingConfig,
|
||||
snapshot: oauthDeletionSnapshot,
|
||||
getTokenUserIds,
|
||||
getUserPrincipals: (candidateUserId) =>
|
||||
db.getUserPrincipals({ userId: candidateUserId }),
|
||||
resolveAllowlists: (candidateUserId) =>
|
||||
registry.resolveAllowlists({ userId: candidateUserId }),
|
||||
fenceAndDisconnectUser: async (candidateUserId) => {
|
||||
if (candidateUserId === userId) {
|
||||
return;
|
||||
}
|
||||
await fenceCommittedMCPMutation({ userId: candidateUserId, serverName });
|
||||
await disconnectLocalMCPServer(candidateUserId, serverName);
|
||||
},
|
||||
uninstallOAuthMCP,
|
||||
});
|
||||
} catch (error) {
|
||||
logger.warn(
|
||||
`[deleteMCPServer] Server ${serverName} was deleted, but OAuth cleanup failed for user ${userId}:`,
|
||||
error,
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
},
|
||||
);
|
||||
if (settled.postWriteError) {
|
||||
throw settled.postWriteError;
|
||||
}
|
||||
res.status(200).json({ message: 'MCP server deleted successfully' });
|
||||
} catch (error) {
|
||||
|
|
|
|||
|
|
@ -137,6 +137,7 @@ jest.mock('~/models', () => ({
|
|||
deleteTokens: jest.fn(),
|
||||
findPluginAuthsByKeys: jest.fn(),
|
||||
getRoleByName: jest.fn(),
|
||||
recordAuditEntry: jest.fn(),
|
||||
}));
|
||||
|
||||
jest.mock('~/server/services/Config', () => ({
|
||||
|
|
@ -4063,6 +4064,63 @@ describe('MCP Routes', () => {
|
|||
expect(MCPConnection.clearCooldown).toHaveBeenCalledWith(inspectionServerName);
|
||||
});
|
||||
|
||||
it('records mcp_server.created with field names and no secret values', async () => {
|
||||
const { recordAuditEntry } = require('~/models');
|
||||
recordAuditEntry.mockClear();
|
||||
const validConfig = {
|
||||
type: 'sse',
|
||||
url: 'https://mcp-server.example.com/sse',
|
||||
title: 'Audited Server',
|
||||
headers: { Authorization: 'Bearer super-secret-token' },
|
||||
};
|
||||
mockRegistryInstance.addServer.mockResolvedValue({
|
||||
serverName: 'audited-server',
|
||||
config: { ...validConfig, source: 'user' },
|
||||
});
|
||||
|
||||
const response = await request(app).post('/api/mcp/servers').send({ config: validConfig });
|
||||
|
||||
expect(response.status).toBe(201);
|
||||
expect(recordAuditEntry).toHaveBeenCalledTimes(1);
|
||||
const input = recordAuditEntry.mock.calls[0][0];
|
||||
expect(input).toEqual(
|
||||
expect.objectContaining({
|
||||
action: 'mcp_server.created',
|
||||
target: { type: 'mcp_server', id: 'audited-server', name: 'Audited Server' },
|
||||
metadata: { source: 'DB', fields: 'headers,title,type,url' },
|
||||
}),
|
||||
);
|
||||
expect(JSON.stringify(input)).not.toContain('super-secret-token');
|
||||
});
|
||||
|
||||
it('audits a create that committed before the cache refresh failed, then returns 500', async () => {
|
||||
const { MCPCommittedWriteError } = require('@librechat/api');
|
||||
const { recordAuditEntry } = require('~/models');
|
||||
recordAuditEntry.mockClear();
|
||||
const validConfig = {
|
||||
type: 'sse',
|
||||
url: 'https://mcp-server.example.com/sse',
|
||||
title: 'Audited Server',
|
||||
};
|
||||
mockRegistryInstance.addServer.mockRejectedValueOnce(
|
||||
new MCPCommittedWriteError(
|
||||
{ serverName: 'audited-server', config: { ...validConfig, source: 'user' } },
|
||||
new Error('redis down'),
|
||||
),
|
||||
);
|
||||
|
||||
const response = await request(app).post('/api/mcp/servers').send({ config: validConfig });
|
||||
|
||||
expect(response.status).toBe(500);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'mcp_server.created',
|
||||
target: { type: 'mcp_server', id: 'audited-server', name: 'Audited Server' },
|
||||
metadata: expect.objectContaining({ postWrite: 'failed' }),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('should reserve config-managed server names when creating MCP server', async () => {
|
||||
const validConfig = {
|
||||
type: 'sse',
|
||||
|
|
@ -4585,6 +4643,93 @@ describe('MCP Routes', () => {
|
|||
);
|
||||
});
|
||||
|
||||
it('records mcp_server.updated with only the changed field names', async () => {
|
||||
const { recordAuditEntry } = require('~/models');
|
||||
recordAuditEntry.mockClear();
|
||||
const existing = {
|
||||
type: 'sse',
|
||||
url: 'https://old.example.com/sse',
|
||||
title: 'Same Title',
|
||||
headers: { Authorization: 'Bearer old-secret' },
|
||||
};
|
||||
const updated = {
|
||||
...existing,
|
||||
url: 'https://new.example.com/sse',
|
||||
headers: { Authorization: 'Bearer new-secret' },
|
||||
};
|
||||
mockRegistryInstance.getServerConfig.mockResolvedValueOnce(existing);
|
||||
mockRegistryInstance.inspectServerUpdate.mockResolvedValue({ ...updated, source: 'user' });
|
||||
mockRegistryInstance.commitServerUpdate.mockResolvedValue({ ...updated, source: 'user' });
|
||||
|
||||
const response = await request(app)
|
||||
.patch('/api/mcp/servers/test-server')
|
||||
.send({ config: updated });
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(recordAuditEntry).toHaveBeenCalledTimes(1);
|
||||
const input = recordAuditEntry.mock.calls[0][0];
|
||||
expect(input).toEqual(
|
||||
expect.objectContaining({
|
||||
action: 'mcp_server.updated',
|
||||
target: { type: 'mcp_server', id: 'test-server', name: 'Same Title' },
|
||||
metadata: { source: 'DB', fields: 'headers,url' },
|
||||
}),
|
||||
);
|
||||
expect(JSON.stringify(input)).not.toContain('secret');
|
||||
});
|
||||
|
||||
it('audits the config this write persisted, not a later overlapping write', async () => {
|
||||
const { recordAuditEntry } = require('~/models');
|
||||
recordAuditEntry.mockClear();
|
||||
const existing = { type: 'sse', url: 'https://old.example.com/sse', title: 'Same Title' };
|
||||
const submitted = { ...existing, url: 'https://new.example.com/sse' };
|
||||
mockRegistryInstance.getServerConfig.mockResolvedValue({
|
||||
...existing,
|
||||
url: 'https://other-writer.example.com/sse',
|
||||
title: 'Other Writer',
|
||||
});
|
||||
mockRegistryInstance.getServerConfig.mockResolvedValueOnce(existing);
|
||||
mockRegistryInstance.inspectServerUpdate.mockResolvedValue({ ...submitted, source: 'user' });
|
||||
mockRegistryInstance.commitServerUpdate.mockResolvedValue({ ...submitted, source: 'user' });
|
||||
|
||||
const response = await request(app)
|
||||
.patch('/api/mcp/servers/test-server')
|
||||
.send({ config: submitted });
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
target: { type: 'mcp_server', id: 'test-server', name: 'Same Title' },
|
||||
metadata: { source: 'DB', fields: 'url' },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('audits an update that committed before the cache refresh failed, then returns 500', async () => {
|
||||
const { MCPCommittedWriteError } = require('@librechat/api');
|
||||
const { recordAuditEntry } = require('~/models');
|
||||
recordAuditEntry.mockClear();
|
||||
const existing = { type: 'sse', url: 'https://old.example.com/sse', title: 'Same Title' };
|
||||
const submitted = { ...existing, url: 'https://new.example.com/sse' };
|
||||
mockRegistryInstance.getServerConfig.mockResolvedValueOnce(existing);
|
||||
mockRegistryInstance.inspectServerUpdate.mockResolvedValue({ ...submitted, source: 'user' });
|
||||
mockRegistryInstance.commitServerUpdate.mockRejectedValueOnce(
|
||||
new MCPCommittedWriteError({ ...submitted, source: 'user' }, new Error('redis down')),
|
||||
);
|
||||
|
||||
const response = await request(app)
|
||||
.patch('/api/mcp/servers/test-server')
|
||||
.send({ config: submitted });
|
||||
|
||||
expect(response.status).toBe(500);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'mcp_server.updated',
|
||||
metadata: { source: 'DB', fields: 'url', postWrite: 'failed' },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('should redact secrets from update response', async () => {
|
||||
const validConfig = {
|
||||
type: 'sse',
|
||||
|
|
@ -4759,6 +4904,56 @@ describe('MCP Routes', () => {
|
|||
);
|
||||
});
|
||||
|
||||
it('records mcp_server.deleted only after the removal succeeds', async () => {
|
||||
const { recordAuditEntry } = require('~/models');
|
||||
recordAuditEntry.mockClear();
|
||||
mockRegistryInstance.removeServer.mockRejectedValueOnce(new Error('Deletion failed'));
|
||||
await request(app).delete('/api/mcp/servers/audited-server');
|
||||
expect(recordAuditEntry).not.toHaveBeenCalled();
|
||||
|
||||
mockRegistryInstance.removeServer.mockResolvedValue(undefined);
|
||||
const response = await request(app).delete('/api/mcp/servers/audited-server');
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'mcp_server.deleted',
|
||||
target: { type: 'mcp_server', id: 'audited-server' },
|
||||
metadata: { source: 'DB' },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('audits a removal that committed before the cache refresh failed, then returns 500', async () => {
|
||||
const { MCPCommittedWriteError } = require('@librechat/api');
|
||||
const { recordAuditEntry } = require('~/models');
|
||||
recordAuditEntry.mockClear();
|
||||
mockRegistryInstance.removeServer.mockRejectedValueOnce(
|
||||
new MCPCommittedWriteError(undefined, new Error('redis down')),
|
||||
);
|
||||
|
||||
const response = await request(app).delete('/api/mcp/servers/audited-server');
|
||||
|
||||
expect(response.status).toBe(500);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'mcp_server.deleted',
|
||||
metadata: { source: 'DB', postWrite: 'failed' },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('does not audit a delete that found nothing to remove', async () => {
|
||||
const { recordAuditEntry } = require('~/models');
|
||||
recordAuditEntry.mockClear();
|
||||
mockRegistryInstance.removeServer.mockResolvedValueOnce(false);
|
||||
|
||||
const response = await request(app).delete('/api/mcp/servers/already-gone');
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(recordAuditEntry).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should return 500 when registry throws error', async () => {
|
||||
mockRegistryInstance.removeServer.mockRejectedValue(new Error('Deletion failed'));
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
const express = require('express');
|
||||
const { createAdminAuditLogHandlers } = require('@librechat/api');
|
||||
const { createAdminAuditLogHandlers, isAuditFailClosed } = require('@librechat/api');
|
||||
const { SystemCapabilities } = require('@librechat/data-schemas');
|
||||
const { requireCapability } = require('~/server/middleware/roles/capabilities');
|
||||
const { requireJwtAuth } = require('~/server/middleware');
|
||||
|
|
@ -15,6 +15,8 @@ const handlers = createAdminAuditLogHandlers({
|
|||
findAuditLogEntry: db.findAuditLogEntry,
|
||||
streamAuditLogEntries: db.streamAuditLogEntries,
|
||||
verifyAuditChain: db.verifyAuditChain,
|
||||
recordAuditEntry: db.recordAuditEntry,
|
||||
auditFailClosed: isAuditFailClosed(),
|
||||
});
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -1,9 +1,10 @@
|
|||
const express = require('express');
|
||||
const { createAdminCodeEnvironmentHandlers } = require('@librechat/api');
|
||||
const { createAdminCodeEnvironmentHandlers, isAuditFailClosed } = require('@librechat/api');
|
||||
const { SystemCapabilities } = require('@librechat/data-schemas');
|
||||
const { requireCapability } = require('~/server/middleware/roles/capabilities');
|
||||
const { getAppConfig } = require('~/server/services/Config');
|
||||
const { requireJwtAuth } = require('~/server/middleware');
|
||||
const db = require('~/models');
|
||||
|
||||
const router = express.Router();
|
||||
const requireAdminAccess = requireCapability(SystemCapabilities.ACCESS_ADMIN);
|
||||
|
|
@ -11,7 +12,11 @@ const requireCodeEnvironmentManage = requireCapability(
|
|||
SystemCapabilities.MANAGE_CODE_ENVIRONMENTS,
|
||||
{ platformOnly: true },
|
||||
);
|
||||
const handlers = createAdminCodeEnvironmentHandlers({ getAppConfig });
|
||||
const handlers = createAdminCodeEnvironmentHandlers({
|
||||
getAppConfig,
|
||||
recordAuditEntry: db.recordAuditEntry,
|
||||
auditFailClosed: isAuditFailClosed(),
|
||||
});
|
||||
|
||||
router.use(requireJwtAuth, requireAdminAccess, requireCodeEnvironmentManage);
|
||||
router.post('/:environmentId/pairings', handlers.createPairing);
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ jest.mock('@librechat/data-schemas', () => ({
|
|||
|
||||
jest.mock('@librechat/api', () => ({
|
||||
createAdminCodeEnvironmentHandlers: jest.fn(() => mockHandlers),
|
||||
isAuditFailClosed: jest.fn(() => true),
|
||||
}));
|
||||
|
||||
jest.mock('~/server/middleware/roles/capabilities', () => ({
|
||||
|
|
@ -44,6 +45,10 @@ jest.mock('~/server/services/Config', () => ({
|
|||
getAppConfig: mockGetAppConfig,
|
||||
}));
|
||||
|
||||
jest.mock('~/models', () => ({
|
||||
recordAuditEntry: jest.fn(),
|
||||
}));
|
||||
|
||||
function createApp() {
|
||||
delete require.cache[require.resolve('./code')];
|
||||
const router = require('./code');
|
||||
|
|
|
|||
|
|
@ -24,6 +24,8 @@ const handlers = createAdminGroupsHandlers({
|
|||
findUsers: db.findUsers,
|
||||
deleteConfig: db.deleteConfig,
|
||||
deleteAclEntries: db.deleteAclEntries,
|
||||
restoreConfig: db.restoreConfig,
|
||||
findConfigByPrincipal: db.findConfigByPrincipal,
|
||||
recordAuditEntry: db.recordAuditEntry,
|
||||
auditFailClosed: process.env.AUDIT_LOG_FAIL_CLOSED === 'true',
|
||||
});
|
||||
|
|
|
|||
|
|
@ -41,6 +41,7 @@ const handlers = createAdminLangfuseHandlers({
|
|||
toggleConfigActive: db.toggleConfigActive,
|
||||
getMessages: db.getMessages,
|
||||
invalidateConfigCaches,
|
||||
recordAuditEntry: db.recordAuditEntry,
|
||||
});
|
||||
|
||||
// `configMiddleware` runs last so unauthorized callers are rejected before the
|
||||
|
|
|
|||
|
|
@ -3,7 +3,11 @@ const { createAdminSkillsSyncAccess, createAdminSkillsSyncHandlers } = require('
|
|||
const { SystemCapabilities } = require('@librechat/data-schemas');
|
||||
const { hasCapability, requireCapability } = require('~/server/middleware/roles/capabilities');
|
||||
const { requireJwtAuth } = require('~/server/middleware');
|
||||
const { upsertSkillSyncCredential, deleteSkillSyncCredential } = require('~/models');
|
||||
const {
|
||||
upsertSkillSyncCredential,
|
||||
deleteSkillSyncCredential,
|
||||
recordAuditEntry,
|
||||
} = require('~/models');
|
||||
const { getGitHubSkillSyncRunnerForRequest } = require('~/server/services/Skills/sync');
|
||||
const { getAppConfig } = require('~/server/services/Config');
|
||||
const configMiddleware = require('~/server/middleware/config/app');
|
||||
|
|
@ -20,6 +24,7 @@ const handlers = createAdminSkillsSyncHandlers({
|
|||
getRunner: getGitHubSkillSyncRunnerForRequest,
|
||||
upsertCredential: upsertSkillSyncCredential,
|
||||
deleteCredential: deleteSkillSyncCredential,
|
||||
recordAuditEntry,
|
||||
});
|
||||
|
||||
router.use(
|
||||
|
|
|
|||
|
|
@ -57,6 +57,7 @@ jest.mock('~/server/services/Config', () => ({
|
|||
jest.mock('~/models', () => ({
|
||||
upsertSkillSyncCredential: jest.fn(),
|
||||
deleteSkillSyncCredential: jest.fn(),
|
||||
recordAuditEntry: jest.fn(),
|
||||
}));
|
||||
|
||||
jest.mock('~/server/services/Skills/sync', () => ({
|
||||
|
|
|
|||
114
e2e/specs/mock/scenarios/admin-audit-integrations.spec.ts
Normal file
114
e2e/specs/mock/scenarios/admin-audit-integrations.spec.ts
Normal file
|
|
@ -0,0 +1,114 @@
|
|||
import { resolve } from 'node:path';
|
||||
import { expect, test } from '@playwright/test';
|
||||
import { run, repoRoot, inOneProject } from './lint.helpers';
|
||||
|
||||
/**
|
||||
* Admin integration changes (MCP servers, Langfuse, audit log reads and
|
||||
* exports, group deletion) leave an audit trail that never carries secrets or
|
||||
* searched-for text, and a group is only removed once its permissions are.
|
||||
* Each probe drives the real handlers from the built @librechat/api with an
|
||||
* in-memory recorder and prints the result as JSON.
|
||||
*/
|
||||
|
||||
const PREAMBLE = `
|
||||
const api = require('@librechat/api');
|
||||
const { Types } = require('mongoose');
|
||||
const out = (value) => { process.stdout.write('RESULT' + JSON.stringify(value)); process.exit(0); };
|
||||
const makeRes = () => {
|
||||
const res = { statusCode: 200, body: undefined, headers: {}, chunks: [] };
|
||||
res.status = (code) => { res.statusCode = code; return res; };
|
||||
res.json = (body) => { res.body = body; return res; };
|
||||
res.setHeader = (key, value) => { res.headers[key] = value; };
|
||||
res.write = (chunk) => { res.chunks.push(String(chunk)); return true; };
|
||||
res.end = () => res;
|
||||
res.once = () => res;
|
||||
res.on = () => res;
|
||||
return res;
|
||||
};
|
||||
const user = { _id: new Types.ObjectId(), name: 'Admin', email: 'admin@example.com' };
|
||||
`;
|
||||
|
||||
function probe(body: string): unknown {
|
||||
const result = run(
|
||||
'node',
|
||||
[
|
||||
'-e',
|
||||
`${PREAMBLE}\n(async () => {\n${body}\n})().catch((e) => { console.error(e); process.exit(1); });`,
|
||||
],
|
||||
{
|
||||
cwd: resolve(repoRoot, 'api'),
|
||||
},
|
||||
);
|
||||
expect(result.status, result.output).toBe(0);
|
||||
return JSON.parse(result.stdout.slice(result.stdout.indexOf('RESULT') + 'RESULT'.length));
|
||||
}
|
||||
|
||||
test.describe('admin integration auditing', () => {
|
||||
test.beforeEach(() => inOneProject());
|
||||
|
||||
test('an MCP server update is audited by field name without the values @scenario:mcp-server-update-audit-lists-field-names-only', () => {
|
||||
const entries = probe(`
|
||||
const entries = [];
|
||||
const audit = api.createMCPServerAuditor({ recordAuditEntry: async (input) => { entries.push(input); } });
|
||||
await audit({ user }, {
|
||||
action: 'mcp_server.updated',
|
||||
serverName: 'srv',
|
||||
previous: { title: 'Docs', url: 'https://old.example.com', headers: { Authorization: 'Bearer old-secret' } },
|
||||
next: { title: 'Docs', url: 'https://old.example.com', headers: { Authorization: 'Bearer new-secret' } },
|
||||
});
|
||||
out(entries);
|
||||
`) as Array<{ action: string; target: { id: string }; metadata: { fields?: string } }>;
|
||||
expect(entries).toHaveLength(1);
|
||||
expect(entries[0].action).toBe('mcp_server.updated');
|
||||
expect(entries[0].target.id).toBe('srv');
|
||||
expect(entries[0].metadata.fields).toBe('headers');
|
||||
expect(JSON.stringify(entries)).not.toContain('secret');
|
||||
});
|
||||
|
||||
test('an MCP server deletion is audited as a warning @scenario:mcp-server-delete-audit-is-a-warning', () => {
|
||||
const entries = probe(`
|
||||
const entries = [];
|
||||
const audit = api.createMCPServerAuditor({ recordAuditEntry: async (input) => { entries.push(input); } });
|
||||
await audit({ user }, { action: 'mcp_server.deleted', serverName: 'gone', previous: { title: 'Gone' } });
|
||||
out(entries);
|
||||
`) as Array<{ action: string; severity: string }>;
|
||||
expect(entries[0].action).toBe('mcp_server.deleted');
|
||||
expect(entries[0].severity).toBe('warning');
|
||||
});
|
||||
|
||||
test('an audit write failure does not turn a deleted MCP server into an error @scenario:audit-write-failure-keeps-the-completed-change', () => {
|
||||
const result = probe(`
|
||||
const audit = api.createMCPServerAuditor({ recordAuditEntry: async () => { throw new Error('db down'); } });
|
||||
let threw = false;
|
||||
try {
|
||||
await audit({ user }, { action: 'mcp_server.created', serverName: 'srv', next: { title: 'Docs' } });
|
||||
} catch { threw = true; }
|
||||
out({ threw });
|
||||
`) as { threw: boolean };
|
||||
expect(result.threw).toBe(false);
|
||||
});
|
||||
|
||||
test('a group whose cleanup fails is kept with its config restored and reports GROUP_CLEANUP_FAILED @scenario:group-delete-keeps-the-group-when-cleanup-fails', () => {
|
||||
const result = probe(`
|
||||
const id = new Types.ObjectId().toString();
|
||||
let deleted = false;
|
||||
const restored = [];
|
||||
const noop = async () => undefined;
|
||||
const handlers = api.createAdminGroupsHandlers({
|
||||
listGroups: noop, countGroups: noop, createGroup: noop, updateGroupById: noop,
|
||||
findGroupById: async () => ({ _id: new Types.ObjectId(id), name: 'G', memberIds: [] }),
|
||||
deleteGroup: async () => { deleted = true; return { _id: id }; },
|
||||
restoreConfig: async () => { restored.push('config'); return true; },
|
||||
deleteConfig: async () => ({ principalType: 'group', principalId: id, overrides: {} }),
|
||||
deleteAclEntries: async () => { throw new Error('acl store down'); },
|
||||
});
|
||||
const res = makeRes();
|
||||
await handlers.deleteGroup({ params: { id }, query: {}, body: {}, user }, res);
|
||||
out({ status: res.statusCode, code: res.body && res.body.code, deleted, restored });
|
||||
`) as { status: number; code?: string; deleted: boolean; restored: string[] };
|
||||
expect(result.status).toBe(500);
|
||||
expect(result.code).toBe('GROUP_CLEANUP_FAILED');
|
||||
expect(result.deleted).toBe(false);
|
||||
expect(result.restored).toEqual(['config']);
|
||||
});
|
||||
});
|
||||
|
|
@ -114,6 +114,7 @@ function createCsvContext(
|
|||
}),
|
||||
end: jest.fn(() => {
|
||||
ended = true;
|
||||
emitter.emit('finish');
|
||||
}),
|
||||
headersSent: false,
|
||||
status: jest.fn().mockReturnValue({ json: jest.fn() }),
|
||||
|
|
@ -351,6 +352,24 @@ describe('createAdminAuditLogHandlers', () => {
|
|||
expect(status).toHaveBeenCalledWith(401);
|
||||
});
|
||||
|
||||
it('records audit.read for an integrity verification', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const handlers = createAdminAuditLogHandlers(createDeps({ recordAuditEntry }));
|
||||
const { req, res } = createReqRes({
|
||||
user: { _id: new Types.ObjectId(), role: 'admin' },
|
||||
});
|
||||
|
||||
await handlers.verifyAuditLog(req, res);
|
||||
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'audit.read',
|
||||
target: { type: 'audit_log' },
|
||||
metadata: { operation: 'verify' },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('returns the verification result scoped to the caller tenant', async () => {
|
||||
const verification = mockVerification({ ok: false, brokenAt: 2, reason: 'hash mismatch' });
|
||||
const deps = createDeps({ verifyAuditChain: jest.fn().mockResolvedValue(verification) });
|
||||
|
|
@ -488,4 +507,189 @@ describe('createAdminAuditLogHandlers', () => {
|
|||
expect(ctx.endCalled()).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('audit entries', () => {
|
||||
const user = { _id: new Types.ObjectId(), role: 'admin' };
|
||||
|
||||
it('records audit.read for a list with filters but not free-text values', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const handlers = createAdminAuditLogHandlers(createDeps({ recordAuditEntry }));
|
||||
const { req, res, status } = createReqRes({
|
||||
user,
|
||||
query: {
|
||||
action: ['role.created', 'group.created'],
|
||||
outcome: 'success',
|
||||
from: '2025-01-01',
|
||||
actorQuery: 'alice@example.com',
|
||||
search: 'secret-term',
|
||||
capability: 'private@example.com',
|
||||
targetType: 'user-typed-text',
|
||||
},
|
||||
});
|
||||
|
||||
await handlers.listAuditLog(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(200);
|
||||
expect(recordAuditEntry).toHaveBeenCalledTimes(1);
|
||||
const input = recordAuditEntry.mock.calls[0][0];
|
||||
expect(input).toEqual(
|
||||
expect.objectContaining({
|
||||
action: 'audit.read',
|
||||
target: { type: 'audit_log' },
|
||||
metadata: {
|
||||
action: 'group.created,role.created',
|
||||
outcome: 'success',
|
||||
from: '2025-01-01T00:00:00.000Z',
|
||||
freeTextFilters: 'search,actorQuery,targetType,capability',
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(JSON.stringify(input)).not.toContain('alice@example.com');
|
||||
expect(JSON.stringify(input)).not.toContain('secret-term');
|
||||
expect(JSON.stringify(input)).not.toContain('private@example.com');
|
||||
expect(JSON.stringify(input)).not.toContain('user-typed-text');
|
||||
});
|
||||
|
||||
it('withholds the rows and answers 500 when a fail-closed audit write fails', async () => {
|
||||
const recordAuditEntry = jest.fn().mockRejectedValue(new Error('audit down'));
|
||||
const listAuditLogPage = jest
|
||||
.fn()
|
||||
.mockResolvedValue({ entries: [mockEntry()], nextCursor: null });
|
||||
const handlers = createAdminAuditLogHandlers(
|
||||
createDeps({ recordAuditEntry, auditFailClosed: true, listAuditLogPage }),
|
||||
);
|
||||
const { req, res, status, json } = createReqRes({ user });
|
||||
|
||||
await handlers.listAuditLog(req, res);
|
||||
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(expect.anything(), { failClosed: true });
|
||||
expect(status).toHaveBeenCalledWith(500);
|
||||
expect(JSON.stringify(json.mock.calls)).not.toContain('entries');
|
||||
});
|
||||
|
||||
it('records audit.read for a single entry only when it exists', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const found = createAdminAuditLogHandlers(
|
||||
createDeps({
|
||||
recordAuditEntry,
|
||||
findAuditLogEntry: jest.fn().mockResolvedValue(mockEntry()),
|
||||
}),
|
||||
);
|
||||
const missing = createAdminAuditLogHandlers(createDeps({ recordAuditEntry }));
|
||||
|
||||
const a = createReqRes({ user, params: { id: validObjectId } });
|
||||
await missing.getAuditLogEntry(a.req, a.res);
|
||||
expect(recordAuditEntry).not.toHaveBeenCalled();
|
||||
|
||||
const b = createReqRes({ user, params: { id: validObjectId } });
|
||||
await found.getAuditLogEntry(b.req, b.res);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'audit.read',
|
||||
target: { type: 'audit_log', id: validObjectId },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('records audit.exported before the first row is written', async () => {
|
||||
const order: string[] = [];
|
||||
const recordAuditEntry = jest.fn().mockImplementation(async () => {
|
||||
order.push('audit');
|
||||
});
|
||||
const deps = createDeps({
|
||||
recordAuditEntry,
|
||||
streamAuditLogEntries: jest.fn().mockImplementation(async () => {
|
||||
order.push('stream');
|
||||
return { count: 0, truncated: false };
|
||||
}),
|
||||
});
|
||||
const handlers = createAdminAuditLogHandlers(deps);
|
||||
const ctx = createCsvContext({ user, query: { severity: 'critical' } });
|
||||
|
||||
await handlers.exportAuditLogCsv(ctx.req, ctx.res);
|
||||
|
||||
expect(order).toEqual(['audit', 'stream', 'audit']);
|
||||
expect(recordAuditEntry).toHaveBeenNthCalledWith(
|
||||
1,
|
||||
expect.objectContaining({
|
||||
action: 'audit.exported',
|
||||
outcome: 'pending',
|
||||
metadata: { format: 'csv', severity: 'critical' },
|
||||
}),
|
||||
);
|
||||
const terminal = recordAuditEntry.mock.calls[recordAuditEntry.mock.calls.length - 1][0];
|
||||
expect(terminal).toEqual(
|
||||
expect.objectContaining({
|
||||
action: 'audit.exported',
|
||||
metadata: { format: 'csv', severity: 'critical', rows: 0, truncated: false },
|
||||
}),
|
||||
);
|
||||
expect(terminal.outcome).toBeUndefined();
|
||||
});
|
||||
|
||||
it('records a client that disconnects during the final flush as a failed export', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const deps = createDeps({
|
||||
recordAuditEntry,
|
||||
streamAuditLogEntries: jest.fn().mockResolvedValue({ count: 0, truncated: false }),
|
||||
});
|
||||
const handlers = createAdminAuditLogHandlers(deps);
|
||||
const ctx = createCsvContext({ user, query: { severity: 'critical' } });
|
||||
(ctx.res.end as jest.Mock).mockImplementation(() => {
|
||||
(ctx.res as unknown as EventEmitter).emit('close');
|
||||
});
|
||||
|
||||
await handlers.exportAuditLogCsv(ctx.req, ctx.res);
|
||||
|
||||
expect(recordAuditEntry).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({
|
||||
outcome: 'failure',
|
||||
metadata: expect.objectContaining({ code: 'client_aborted' }),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('appends a failure entry when the export fails after it was recorded', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const deps = createDeps({
|
||||
recordAuditEntry,
|
||||
streamAuditLogEntries: jest.fn().mockRejectedValue(new Error('cursor failed')),
|
||||
});
|
||||
const handlers = createAdminAuditLogHandlers(deps);
|
||||
const ctx = createCsvContext({ user, query: { severity: 'critical' } });
|
||||
|
||||
await handlers.exportAuditLogCsv(ctx.req, ctx.res);
|
||||
|
||||
expect(recordAuditEntry).toHaveBeenCalledTimes(2);
|
||||
expect(recordAuditEntry).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'audit.exported',
|
||||
outcome: 'failure',
|
||||
metadata: { format: 'csv', severity: 'critical', code: 'export_failed' },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('does not record an export that fails filter validation', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const handlers = createAdminAuditLogHandlers(createDeps({ recordAuditEntry }));
|
||||
const ctx = createCsvContext({ user, query: { action: 'nope.nope' } });
|
||||
|
||||
await handlers.exportAuditLogCsv(ctx.req, ctx.res);
|
||||
|
||||
expect(recordAuditEntry).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('blocks the export when failClosed and the audit write fails', async () => {
|
||||
const recordAuditEntry = jest.fn().mockRejectedValue(new Error('audit down'));
|
||||
const deps = createDeps({ recordAuditEntry, auditFailClosed: true });
|
||||
const handlers = createAdminAuditLogHandlers(deps);
|
||||
const ctx = createCsvContext({ user });
|
||||
|
||||
await handlers.exportAuditLogCsv(ctx.req, ctx.res);
|
||||
|
||||
expect(deps.streamAuditLogEntries).not.toHaveBeenCalled();
|
||||
expect(ctx.chunks).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -20,8 +20,12 @@ import type {
|
|||
AuditOutcome,
|
||||
AuditSeverity,
|
||||
} from '@librechat/data-schemas';
|
||||
import type { AuditMetadata } from '@librechat/data-schemas';
|
||||
import type { Response } from 'express';
|
||||
import type { ServerRequest } from '~/types/http';
|
||||
import type { RecordAuditEntry } from './audit';
|
||||
import { getSafeErrorMetadata } from '~/utils/errors';
|
||||
import { createAdminAuditEmitter } from './audit';
|
||||
|
||||
const FORMULA_PREFIX = /^[=+\-@\t\r]/;
|
||||
/** UTF-8 BOM, written first so Excel recognizes the encoding on import. Spelled
|
||||
|
|
@ -64,6 +68,9 @@ export interface AdminAuditLogDeps {
|
|||
tenantId: string | undefined,
|
||||
options?: { isCancelled?: () => boolean; maxRows?: number },
|
||||
) => Promise<AuditChainVerification>;
|
||||
/** Optional audit emission for reads and exports; fail-open unless `auditFailClosed`. */
|
||||
recordAuditEntry?: RecordAuditEntry;
|
||||
auditFailClosed?: boolean;
|
||||
}
|
||||
|
||||
interface CallerContext {
|
||||
|
|
@ -270,6 +277,32 @@ function parseFilters(query: AuditLogQuery): ParseResult<ParsedFilters> {
|
|||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Describes a filter set for the audit trail without storing what was searched
|
||||
* for: enum and date filters are recorded as given, free-text filters (which can
|
||||
* carry names or email addresses) only by name.
|
||||
*/
|
||||
function describeFilters(filters: ParsedFilters): AuditMetadata {
|
||||
const metadata: AuditMetadata = {};
|
||||
const lists: Array<[string, string[] | undefined]> = [
|
||||
['category', filters.category],
|
||||
['action', filters.action],
|
||||
['outcome', filters.outcome],
|
||||
['severity', filters.severity],
|
||||
];
|
||||
for (const [key, values] of lists) {
|
||||
if (values?.length) metadata[key] = [...values].sort().join(',');
|
||||
}
|
||||
if (filters.actorType) metadata.actorType = filters.actorType;
|
||||
if (filters.from) metadata.from = filters.from.toISOString();
|
||||
if (filters.to) metadata.to = filters.to.toISOString();
|
||||
const freeText = (
|
||||
['search', 'actorQuery', 'targetQuery', 'targetType', 'capability'] as const
|
||||
).filter((key) => filters[key] != null);
|
||||
if (freeText.length > 0) metadata.freeTextFilters = freeText.join(',');
|
||||
return metadata;
|
||||
}
|
||||
|
||||
export function createAdminAuditLogHandlers(deps: AdminAuditLogDeps): {
|
||||
listAuditLog: (req: ServerRequest, res: Response) => Promise<Response>;
|
||||
getAuditLogEntry: (req: ServerRequest, res: Response) => Promise<Response>;
|
||||
|
|
@ -277,6 +310,17 @@ export function createAdminAuditLogHandlers(deps: AdminAuditLogDeps): {
|
|||
exportAuditLogCsv: (req: ServerRequest, res: Response) => Promise<Response | void>;
|
||||
} {
|
||||
const { listAuditLogPage, findAuditLogEntry, streamAuditLogEntries, verifyAuditChain } = deps;
|
||||
const emitAudit = createAdminAuditEmitter({
|
||||
recordAuditEntry: deps.recordAuditEntry,
|
||||
failClosed: deps.auditFailClosed,
|
||||
logPrefix: '[adminAuditLog]',
|
||||
});
|
||||
|
||||
/** Follow-up entries describe an attempt that already happened, so they never fail the caller. */
|
||||
const emitFollowUp = createAdminAuditEmitter({
|
||||
recordAuditEntry: deps.recordAuditEntry,
|
||||
logPrefix: '[adminAuditLog]',
|
||||
});
|
||||
|
||||
async function listAuditLogHandler(req: ServerRequest, res: Response) {
|
||||
try {
|
||||
|
|
@ -301,9 +345,14 @@ export function createAdminAuditLogHandlers(deps: AdminAuditLogDeps): {
|
|||
cursor: cursor.value,
|
||||
});
|
||||
|
||||
await emitAudit(req, {
|
||||
action: 'audit.read',
|
||||
target: { type: 'audit_log' },
|
||||
metadata: describeFilters(filters.value),
|
||||
});
|
||||
return res.status(200).json(page);
|
||||
} catch (err) {
|
||||
logger.error('[adminAuditLog] list error:', err);
|
||||
logger.error('[adminAuditLog] list error:', getSafeErrorMetadata(err));
|
||||
return res.status(500).json({ error: 'Failed to fetch audit log' });
|
||||
}
|
||||
}
|
||||
|
|
@ -320,9 +369,13 @@ export function createAdminAuditLogHandlers(deps: AdminAuditLogDeps): {
|
|||
|
||||
const entry = await findAuditLogEntry(caller.tenantId, id);
|
||||
if (!entry) return res.status(404).json({ error: 'Not found' });
|
||||
await emitAudit(req, {
|
||||
action: 'audit.read',
|
||||
target: { type: 'audit_log', id },
|
||||
});
|
||||
return res.status(200).json({ entry });
|
||||
} catch (err) {
|
||||
logger.error('[adminAuditLog] get error:', err);
|
||||
logger.error('[adminAuditLog] get error:', getSafeErrorMetadata(err));
|
||||
return res.status(500).json({ error: 'Failed to fetch audit log entry' });
|
||||
}
|
||||
}
|
||||
|
|
@ -346,18 +399,25 @@ export function createAdminAuditLogHandlers(deps: AdminAuditLogDeps): {
|
|||
});
|
||||
|
||||
if (clientAborted) return res;
|
||||
await emitAudit(req, {
|
||||
action: 'audit.read',
|
||||
target: { type: 'audit_log' },
|
||||
metadata: { operation: 'verify' },
|
||||
});
|
||||
return res.status(200).json(result);
|
||||
} finally {
|
||||
res.removeListener('close', markAborted);
|
||||
req.removeListener('aborted', markAborted);
|
||||
}
|
||||
} catch (err) {
|
||||
logger.error('[adminAuditLog] verify error:', err);
|
||||
logger.error('[adminAuditLog] verify error:', getSafeErrorMetadata(err));
|
||||
return res.status(500).json({ error: 'Failed to verify audit log integrity' });
|
||||
}
|
||||
}
|
||||
|
||||
async function exportAuditLogCsvHandler(req: ServerRequest, res: Response) {
|
||||
let exportRecorded = false;
|
||||
let exportFailed: ((code: string) => Promise<void>) | undefined;
|
||||
try {
|
||||
const caller = resolveCaller(req);
|
||||
if (!caller) return res.status(401).json({ error: 'Authentication required' });
|
||||
|
|
@ -365,11 +425,6 @@ export function createAdminAuditLogHandlers(deps: AdminAuditLogDeps): {
|
|||
const filters = parseFilters(req.query as AuditLogQuery);
|
||||
if (!filters.ok) return res.status(400).json({ error: filters.error });
|
||||
|
||||
const filenameStamp = new Date().toISOString().slice(0, 10);
|
||||
res.setHeader('Content-Type', 'text/csv; charset=utf-8');
|
||||
res.setHeader('Content-Disposition', `attachment; filename="audit-log-${filenameStamp}.csv"`);
|
||||
res.setHeader('Cache-Control', 'no-store');
|
||||
|
||||
/**
|
||||
* The socket-`close` listener is the canonical signal — it fires on client
|
||||
* TCP RST as well as on graceful end. `req.aborted` (deprecated) is kept as
|
||||
|
|
@ -384,6 +439,27 @@ export function createAdminAuditLogHandlers(deps: AdminAuditLogDeps): {
|
|||
req.once('aborted', markAborted);
|
||||
const isCancelled = () => clientAborted;
|
||||
|
||||
/** Recorded as pending before any row is streamed, so an export is never unaudited; `success` follows once the stream ends. */
|
||||
const exportEvent = {
|
||||
action: 'audit.exported' as const,
|
||||
severity: 'warning' as const,
|
||||
target: { type: 'audit_log' },
|
||||
metadata: { format: 'csv', ...describeFilters(filters.value) },
|
||||
};
|
||||
await emitAudit(req, { ...exportEvent, outcome: 'pending' });
|
||||
exportRecorded = true;
|
||||
exportFailed = (code: string) =>
|
||||
emitFollowUp(req, {
|
||||
...exportEvent,
|
||||
outcome: 'failure',
|
||||
metadata: { ...exportEvent.metadata, code },
|
||||
});
|
||||
|
||||
const filenameStamp = new Date().toISOString().slice(0, 10);
|
||||
res.setHeader('Content-Type', 'text/csv; charset=utf-8');
|
||||
res.setHeader('Content-Disposition', `attachment; filename="audit-log-${filenameStamp}.csv"`);
|
||||
res.setHeader('Cache-Control', 'no-store');
|
||||
|
||||
/**
|
||||
* Wait for `drain` when the socket buffer is full so we never queue an
|
||||
* unbounded amount of CSV in Node memory for slow consumers. Race against
|
||||
|
|
@ -411,7 +487,7 @@ export function createAdminAuditLogHandlers(deps: AdminAuditLogDeps): {
|
|||
await writeChunk(formatCsvHeader());
|
||||
await writeChunk('\r\n');
|
||||
|
||||
const { truncated } = await streamAuditLogEntries(
|
||||
const { truncated, count } = await streamAuditLogEntries(
|
||||
caller.tenantId,
|
||||
filters.value,
|
||||
async (entry) => {
|
||||
|
|
@ -424,7 +500,19 @@ export function createAdminAuditLogHandlers(deps: AdminAuditLogDeps): {
|
|||
|
||||
res.removeListener('close', markAborted);
|
||||
req.removeListener('aborted', markAborted);
|
||||
if (clientAborted) {
|
||||
await exportFailed('client_aborted');
|
||||
}
|
||||
if (!clientAborted) {
|
||||
/** `end()` only queues the last bytes, so success waits for the response to finish. */
|
||||
const settled = new Promise<boolean>((resolve) => {
|
||||
if (res.destroyed) {
|
||||
resolve(false);
|
||||
return;
|
||||
}
|
||||
res.once('finish', () => resolve(true));
|
||||
res.once('close', () => resolve(res.writableFinished === true));
|
||||
});
|
||||
/**
|
||||
* `truncated` is true only when rows existed beyond the cap (an exact-cap
|
||||
* match is reported as complete). Surfacing this is a compliance
|
||||
|
|
@ -441,9 +529,20 @@ export function createAdminAuditLogHandlers(deps: AdminAuditLogDeps): {
|
|||
);
|
||||
}
|
||||
res.end();
|
||||
if (await settled) {
|
||||
await emitFollowUp(req, {
|
||||
...exportEvent,
|
||||
metadata: { ...exportEvent.metadata, rows: count, truncated },
|
||||
});
|
||||
} else {
|
||||
await exportFailed('client_aborted');
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
logger.error('[adminAuditLog] export error:', err);
|
||||
logger.error('[adminAuditLog] export error:', getSafeErrorMetadata(err));
|
||||
if (exportRecorded) {
|
||||
await exportFailed?.('export_failed');
|
||||
}
|
||||
if (!res.headersSent) {
|
||||
return res.status(500).json({ error: 'Failed to export audit log' });
|
||||
}
|
||||
|
|
|
|||
|
|
@ -339,4 +339,194 @@ describe('createAdminCodeEnvironmentHandlers', () => {
|
|||
expect(response.statusCode).toBe(502);
|
||||
expect(response.body).toEqual({ error: 'Code API returned an invalid revocation response' });
|
||||
});
|
||||
|
||||
describe('audit entries', () => {
|
||||
function build(fetchImpl: jest.Mock, recordAuditEntry: jest.Mock) {
|
||||
return createAdminCodeEnvironmentHandlers({
|
||||
getAppConfig: jest.fn().mockResolvedValue(config()),
|
||||
readSecret: jest.fn().mockReturnValue('administrator-bootstrap-token'),
|
||||
fetchImpl,
|
||||
recordAuditEntry,
|
||||
});
|
||||
}
|
||||
|
||||
it('records code_worker.paired without the pairing code', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const fetchImpl = jest.fn().mockResolvedValue(
|
||||
Response.json({
|
||||
protocolVersion: 1,
|
||||
workerId: 'vm-1',
|
||||
code: 'one-time-code-value-that-is-long',
|
||||
expiresAt: '2099-08-30T12:00:00.000Z',
|
||||
}),
|
||||
);
|
||||
|
||||
await build(fetchImpl, recordAuditEntry).createPairing(request(), mockResponse());
|
||||
|
||||
expect(recordAuditEntry.mock.calls.map(([entry]) => entry.outcome ?? 'success')).toEqual([
|
||||
'pending',
|
||||
'success',
|
||||
]);
|
||||
const input = recordAuditEntry.mock.calls[1][0];
|
||||
expect(input).toEqual(
|
||||
expect.objectContaining({
|
||||
action: 'code_worker.paired',
|
||||
target: { type: 'code_worker', id: 'vm-1', name: 'attached-vm' },
|
||||
}),
|
||||
);
|
||||
expect(input.tenantId).toBeUndefined();
|
||||
expect(input.metadata.boundTenantId).toEqual(expect.any(String));
|
||||
expect(JSON.stringify(recordAuditEntry.mock.calls)).not.toContain(
|
||||
'one-time-code-value-that-is-long',
|
||||
);
|
||||
expect(JSON.stringify(recordAuditEntry.mock.calls)).not.toContain(
|
||||
'administrator-bootstrap-token',
|
||||
);
|
||||
});
|
||||
|
||||
it('records code_worker.revoked as pending, then success or a failure entry when it is not confirmed', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const ok = jest.fn().mockResolvedValue(Response.json({ protocolVersion: 1, revoked: true }));
|
||||
const bad = jest
|
||||
.fn()
|
||||
.mockResolvedValue(Response.json({ protocolVersion: 1, revoked: false }));
|
||||
|
||||
await build(ok, recordAuditEntry).revokeWorker(request(), mockResponse());
|
||||
expect(recordAuditEntry).toHaveBeenCalledTimes(2);
|
||||
expect(recordAuditEntry).toHaveBeenNthCalledWith(
|
||||
1,
|
||||
expect.objectContaining({
|
||||
action: 'code_worker.revoked',
|
||||
outcome: 'pending',
|
||||
target: { type: 'code_worker', id: 'vm-1', name: 'attached-vm' },
|
||||
}),
|
||||
);
|
||||
expect(recordAuditEntry).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({ action: 'code_worker.revoked', outcome: 'success' }),
|
||||
);
|
||||
|
||||
recordAuditEntry.mockClear();
|
||||
await build(bad, recordAuditEntry).revokeWorker(request(), mockResponse());
|
||||
expect(recordAuditEntry).toHaveBeenCalledTimes(2);
|
||||
expect(recordAuditEntry).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({
|
||||
outcome: 'failure',
|
||||
metadata: { environmentId: 'attached-vm', code: 'invalid_response' },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('records the pairing before the bridge call and a failure entry when no code is minted', async () => {
|
||||
const order: string[] = [];
|
||||
const recordAuditEntry = jest.fn().mockImplementation(async (input) => {
|
||||
order.push(`audit:${input.outcome ?? 'success'}`);
|
||||
});
|
||||
const fetchImpl = jest.fn().mockImplementation(async () => {
|
||||
order.push('bridge');
|
||||
return new Response('nope', { status: 500 });
|
||||
});
|
||||
|
||||
await build(fetchImpl, recordAuditEntry).createPairing(request(), mockResponse());
|
||||
|
||||
expect(order).toEqual(['audit:pending', 'bridge', 'audit:failure']);
|
||||
expect(recordAuditEntry).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({ metadata: expect.objectContaining({ code: 'bridge_rejected' }) }),
|
||||
);
|
||||
});
|
||||
|
||||
it('sends the revocation without waiting for a slow pending audit write', async () => {
|
||||
const order: string[] = [];
|
||||
let release: () => void = () => undefined;
|
||||
const recordAuditEntry = jest.fn().mockImplementation(async (input) => {
|
||||
if (input.outcome === 'pending') {
|
||||
await new Promise<void>((resolve) => (release = resolve));
|
||||
}
|
||||
order.push(`audit:${input.outcome ?? 'success'}`);
|
||||
});
|
||||
const fetchImpl = jest.fn().mockImplementation(async () => {
|
||||
order.push('bridge');
|
||||
release();
|
||||
return Response.json({ protocolVersion: 1, revoked: true });
|
||||
});
|
||||
|
||||
await build(fetchImpl, recordAuditEntry).revokeWorker(request(), mockResponse());
|
||||
|
||||
expect(order).toEqual(['bridge', 'audit:pending', 'audit:success']);
|
||||
});
|
||||
|
||||
it('delivers the pairing code before the terminal audit write settles', async () => {
|
||||
let release: () => void = () => undefined;
|
||||
let deliveredAtTerminalWrite: unknown;
|
||||
const response = mockResponse();
|
||||
const recordAuditEntry = jest.fn().mockImplementation(async (input) => {
|
||||
if (input.outcome === undefined || input.outcome === 'success') {
|
||||
deliveredAtTerminalWrite = response.body;
|
||||
await new Promise<void>((resolve) => (release = resolve));
|
||||
}
|
||||
});
|
||||
const fetchImpl = jest.fn().mockResolvedValue(
|
||||
Response.json({
|
||||
protocolVersion: 1,
|
||||
workerId: 'vm-1',
|
||||
code: 'one-time-code-value-that-is-long',
|
||||
expiresAt: '2099-08-30T12:00:00.000Z',
|
||||
}),
|
||||
);
|
||||
|
||||
const pending = build(fetchImpl, recordAuditEntry).createPairing(request(), response);
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
release();
|
||||
await pending;
|
||||
expect(deliveredAtTerminalWrite).toEqual(expect.objectContaining({ workerId: 'vm-1' }));
|
||||
});
|
||||
|
||||
it('does not pair a worker when the pending audit write fails under fail-closed', async () => {
|
||||
const fetchImpl = jest.fn();
|
||||
const response = mockResponse();
|
||||
|
||||
await createAdminCodeEnvironmentHandlers({
|
||||
getAppConfig: jest.fn().mockResolvedValue(config()),
|
||||
readSecret: jest.fn().mockReturnValue('administrator-bootstrap-token'),
|
||||
fetchImpl,
|
||||
recordAuditEntry: jest.fn().mockRejectedValue(new Error('audit down')),
|
||||
auditFailClosed: true,
|
||||
})
|
||||
.createPairing(request(), response)
|
||||
.catch(() => undefined);
|
||||
|
||||
expect(fetchImpl).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('never blocks a revocation on an audit outage, even when fail-closed', async () => {
|
||||
const ok = jest.fn().mockResolvedValue(Response.json({ protocolVersion: 1, revoked: true }));
|
||||
const response = mockResponse();
|
||||
|
||||
await createAdminCodeEnvironmentHandlers({
|
||||
getAppConfig: jest.fn().mockResolvedValue(config()),
|
||||
readSecret: jest.fn().mockReturnValue('administrator-bootstrap-token'),
|
||||
fetchImpl: ok,
|
||||
recordAuditEntry: jest.fn().mockRejectedValue(new Error('audit down')),
|
||||
auditFailClosed: true,
|
||||
}).revokeWorker(request(), response);
|
||||
|
||||
expect(response.statusCode).toBe(200);
|
||||
});
|
||||
|
||||
it('does not fail the pairing when the audit write fails (fail-open)', async () => {
|
||||
const recordAuditEntry = jest.fn().mockRejectedValue(new Error('audit down'));
|
||||
const fetchImpl = jest.fn().mockResolvedValue(
|
||||
Response.json({
|
||||
protocolVersion: 1,
|
||||
workerId: 'vm-1',
|
||||
code: 'one-time-code-value-that-is-long',
|
||||
expiresAt: '2099-08-30T12:00:00.000Z',
|
||||
}),
|
||||
);
|
||||
const response = mockResponse();
|
||||
|
||||
await build(fetchImpl, recordAuditEntry).createPairing(request(), response);
|
||||
|
||||
expect(response.statusCode).toBe(200);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -4,8 +4,10 @@ import type { AppConfig } from '@librechat/data-schemas';
|
|||
import type { Response } from 'express';
|
||||
import type { GetAppConfigOptions } from '~/app/service';
|
||||
import type { ServerRequest } from '~/types/http';
|
||||
import type { RecordAuditEntry } from './audit';
|
||||
|
||||
import { getCodeApiTenantId } from '~/auth/codeapi';
|
||||
import { createAdminAuditEmitter } from './audit';
|
||||
|
||||
const CODE_BRIDGE_REQUEST_TIMEOUT_MS = 10_000;
|
||||
|
||||
|
|
@ -36,6 +38,9 @@ export interface AdminCodeEnvironmentDeps {
|
|||
resolveTenantId?: (req: ServerRequest) => string;
|
||||
readSecret?: (name: string) => string | undefined;
|
||||
fetchImpl?: FetchImpl;
|
||||
/** Optional audit emission for pairing and revocation; fail-open unless `auditFailClosed`. */
|
||||
recordAuditEntry?: RecordAuditEntry;
|
||||
auditFailClosed?: boolean;
|
||||
}
|
||||
|
||||
function environmentId(req: ServerRequest): string {
|
||||
|
|
@ -95,6 +100,17 @@ export function createAdminCodeEnvironmentHandlers(deps: AdminCodeEnvironmentDep
|
|||
} {
|
||||
const fetchImpl = deps.fetchImpl ?? fetch;
|
||||
const resolveTenantId = deps.resolveTenantId ?? getCodeApiTenantId;
|
||||
const emitAudit = createAdminAuditEmitter({
|
||||
recordAuditEntry: deps.recordAuditEntry,
|
||||
failClosed: deps.auditFailClosed,
|
||||
logPrefix: '[adminCode]',
|
||||
});
|
||||
/** Revoking a worker removes access, and terminal entries record an outcome that already
|
||||
* happened, so an audit outage blocks neither. */
|
||||
const emitRevocationAudit = createAdminAuditEmitter({
|
||||
recordAuditEntry: deps.recordAuditEntry,
|
||||
logPrefix: '[adminCode]',
|
||||
});
|
||||
const readSecret =
|
||||
deps.readSecret ??
|
||||
((name: string) =>
|
||||
|
|
@ -144,6 +160,28 @@ export function createAdminCodeEnvironmentHandlers(deps: AdminCodeEnvironmentDep
|
|||
} catch {
|
||||
return res.status(503).json({ error: 'Code environment tenant context is unavailable' });
|
||||
}
|
||||
/**
|
||||
* Recorded as `pending` before the bridge mints the one-time code, so no audit latency
|
||||
* sits between minting and delivering it and a rejected pairing is never counted as
|
||||
* granted; `success` is appended once the bridge response validates and `failure`
|
||||
* otherwise. Workers belong to the deployment's YAML environments, so entries go to
|
||||
* the platform chain.
|
||||
*/
|
||||
const pairingEvent = {
|
||||
action: 'code_worker.paired' as const,
|
||||
severity: 'warning' as const,
|
||||
target: { type: 'code_worker', id: resolved.pairing.workerId, name: resolved.id },
|
||||
tenantId: undefined,
|
||||
};
|
||||
const pairingMetadata = { environmentId: resolved.id, boundTenantId: tenantId };
|
||||
await emitAudit(req, { ...pairingEvent, outcome: 'pending', metadata: pairingMetadata });
|
||||
const pairingFailed = async (code: string) => {
|
||||
await emitRevocationAudit(req, {
|
||||
...pairingEvent,
|
||||
outcome: 'failure',
|
||||
metadata: { ...pairingMetadata, code },
|
||||
});
|
||||
};
|
||||
try {
|
||||
const response = await fetchImpl(bridgeUrl(resolved.environment, '/bridge/pairings'), {
|
||||
method: 'POST',
|
||||
|
|
@ -162,6 +200,7 @@ export function createAdminCodeEnvironmentHandlers(deps: AdminCodeEnvironmentDep
|
|||
signal: AbortSignal.timeout(CODE_BRIDGE_REQUEST_TIMEOUT_MS),
|
||||
});
|
||||
if (!response.ok) {
|
||||
await pairingFailed('bridge_rejected');
|
||||
return res.status(502).json({
|
||||
error: 'Code API rejected the pairing request',
|
||||
upstreamStatus: response.status,
|
||||
|
|
@ -169,16 +208,21 @@ export function createAdminCodeEnvironmentHandlers(deps: AdminCodeEnvironmentDep
|
|||
}
|
||||
const payload = (await response.json()) as unknown;
|
||||
if (!validPairingResponse(payload, resolved.pairing.workerId)) {
|
||||
await pairingFailed('invalid_response');
|
||||
return res.status(502).json({ error: 'Code API returned an invalid pairing response' });
|
||||
}
|
||||
return res.status(200).json({
|
||||
/** The code is delivered first: it expires, so audit latency must not sit in front of it. */
|
||||
res.status(200).json({
|
||||
environmentId: resolved.id,
|
||||
workerId: payload.workerId,
|
||||
code: payload.code,
|
||||
expiresAt: payload.expiresAt,
|
||||
});
|
||||
await emitRevocationAudit(req, { ...pairingEvent, metadata: pairingMetadata });
|
||||
return res;
|
||||
} catch (error) {
|
||||
const timedOut = error instanceof Error && error.name === 'TimeoutError';
|
||||
await pairingFailed(timedOut ? 'timeout' : 'request_failed');
|
||||
return res.status(timedOut ? 504 : 502).json({
|
||||
error: timedOut ? 'Code API pairing request timed out' : 'Code API pairing request failed',
|
||||
});
|
||||
|
|
@ -188,6 +232,32 @@ export function createAdminCodeEnvironmentHandlers(deps: AdminCodeEnvironmentDep
|
|||
async function revokeWorker(req: ServerRequest, res: Response): Promise<Response> {
|
||||
const resolved = await resolve(req, res);
|
||||
if ('statusCode' in resolved) return resolved;
|
||||
/**
|
||||
* Recorded as `pending` without being awaited, so audit latency never delays the
|
||||
* revocation; later entries wait for it to keep the chain ordered. `success` is
|
||||
* appended once the bridge acknowledges and `failure` otherwise.
|
||||
*/
|
||||
const revokeEvent = {
|
||||
action: 'code_worker.revoked' as const,
|
||||
severity: 'warning' as const,
|
||||
target: { type: 'code_worker', id: resolved.pairing.workerId, name: resolved.id },
|
||||
tenantId: undefined,
|
||||
};
|
||||
const revokeMetadata = { environmentId: resolved.id };
|
||||
const attempt = emitRevocationAudit(req, {
|
||||
...revokeEvent,
|
||||
outcome: 'pending',
|
||||
metadata: revokeMetadata,
|
||||
});
|
||||
const revokeSettled = async (outcome: 'success' | 'failure', code?: string) => {
|
||||
await attempt;
|
||||
await emitRevocationAudit(req, {
|
||||
...revokeEvent,
|
||||
outcome,
|
||||
metadata: code ? { ...revokeMetadata, code } : revokeMetadata,
|
||||
});
|
||||
};
|
||||
const revokeFailed = (code: string) => revokeSettled('failure', code);
|
||||
try {
|
||||
const workerId = encodeURIComponent(resolved.pairing.workerId);
|
||||
const response = await fetchImpl(
|
||||
|
|
@ -204,6 +274,7 @@ export function createAdminCodeEnvironmentHandlers(deps: AdminCodeEnvironmentDep
|
|||
},
|
||||
);
|
||||
if (!response.ok) {
|
||||
await revokeFailed('bridge_rejected');
|
||||
return res.status(502).json({
|
||||
error: 'Code API rejected the revocation request',
|
||||
upstreamStatus: response.status,
|
||||
|
|
@ -211,15 +282,19 @@ export function createAdminCodeEnvironmentHandlers(deps: AdminCodeEnvironmentDep
|
|||
}
|
||||
const payload = (await response.json()) as unknown;
|
||||
if (!validRevocationResponse(payload)) {
|
||||
await revokeFailed('invalid_response');
|
||||
return res.status(502).json({ error: 'Code API returned an invalid revocation response' });
|
||||
}
|
||||
return res.status(200).json({
|
||||
res.status(200).json({
|
||||
environmentId: resolved.id,
|
||||
workerId: resolved.pairing.workerId,
|
||||
revoked: true,
|
||||
});
|
||||
await revokeSettled('success');
|
||||
return res;
|
||||
} catch (error) {
|
||||
const timedOut = error instanceof Error && error.name === 'TimeoutError';
|
||||
await revokeFailed(timedOut ? 'timeout' : 'request_failed');
|
||||
return res.status(timedOut ? 504 : 502).json({
|
||||
error: timedOut
|
||||
? 'Code API revocation request timed out'
|
||||
|
|
|
|||
19
packages/api/src/admin/context.spec.ts
Normal file
19
packages/api/src/admin/context.spec.ts
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
import type { ServerRequest } from '~/types/http';
|
||||
import { buildAuditContext } from './context';
|
||||
|
||||
describe('buildAuditContext', () => {
|
||||
it('prefers the server-assigned request id over client headers', () => {
|
||||
const req = {
|
||||
requestId: 'server-id',
|
||||
headers: { 'x-request-id': 'client-id' },
|
||||
} as unknown as ServerRequest;
|
||||
|
||||
expect(buildAuditContext(req).requestId).toBe('server-id');
|
||||
});
|
||||
|
||||
it('falls back to the correlation headers', () => {
|
||||
const req = { headers: { 'x-correlation-id': 'client-id' } } as unknown as ServerRequest;
|
||||
|
||||
expect(buildAuditContext(req).requestId).toBe('client-id');
|
||||
});
|
||||
});
|
||||
|
|
@ -8,7 +8,7 @@ function firstHeaderValue(value: string | string[] | undefined): string | undefi
|
|||
}
|
||||
|
||||
/** Extracts forensic request context (IP, user agent, correlation id) for an
|
||||
* audit record. Fields are undefined when the data isn't available. */
|
||||
* audit record. The server-assigned request id wins over client-supplied headers. Fields are undefined when the data isn't available. */
|
||||
export function buildAuditContext(req: ServerRequest): AuditContext {
|
||||
const headers = req.headers ?? {};
|
||||
const forwarded = firstHeaderValue(headers['x-forwarded-for']);
|
||||
|
|
@ -16,6 +16,8 @@ export function buildAuditContext(req: ServerRequest): AuditContext {
|
|||
return {
|
||||
ip: req.ip || forwardedIp || req.socket?.remoteAddress || undefined,
|
||||
userAgent: firstHeaderValue(headers['user-agent']),
|
||||
requestId: firstHeaderValue(headers['x-request-id'] ?? headers['x-correlation-id']),
|
||||
requestId:
|
||||
(req as { requestId?: string }).requestId ??
|
||||
firstHeaderValue(headers['x-request-id'] ?? headers['x-correlation-id']),
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -812,7 +812,10 @@ describe('createAdminGroupsHandlers', () => {
|
|||
|
||||
describe('deleteGroup', () => {
|
||||
it('deletes group and returns 200 with id', async () => {
|
||||
const deps = createDeps({ deleteGroup: jest.fn().mockResolvedValue(mockGroup()) });
|
||||
const deps = createDeps({
|
||||
findGroupById: jest.fn().mockResolvedValue(mockGroup()),
|
||||
deleteGroup: jest.fn().mockResolvedValue(mockGroup()),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status, json } = createReqRes({ params: { id: validId } });
|
||||
|
||||
|
|
@ -834,8 +837,8 @@ describe('createAdminGroupsHandlers', () => {
|
|||
expect(json).toHaveBeenCalledWith({ error: 'Invalid group ID format' });
|
||||
});
|
||||
|
||||
it('returns 404 when deleteGroup returns null', async () => {
|
||||
const deps = createDeps({ deleteGroup: jest.fn().mockResolvedValue(null) });
|
||||
it('returns 404 and cleans nothing when the group is missing', async () => {
|
||||
const deps = createDeps({ findGroupById: jest.fn().mockResolvedValue(null) });
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status, json } = createReqRes({ params: { id: validId } });
|
||||
|
||||
|
|
@ -844,10 +847,12 @@ describe('createAdminGroupsHandlers', () => {
|
|||
expect(status).toHaveBeenCalledWith(404);
|
||||
expect(json).toHaveBeenCalledWith({ error: 'Group not found' });
|
||||
expect(deps.deleteConfig).not.toHaveBeenCalled();
|
||||
expect(deps.deleteGroup).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('returns 500 on error', async () => {
|
||||
const deps = createDeps({
|
||||
findGroupById: jest.fn().mockResolvedValue(mockGroup()),
|
||||
deleteGroup: jest.fn().mockRejectedValue(new Error('db down')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
|
|
@ -859,9 +864,9 @@ describe('createAdminGroupsHandlers', () => {
|
|||
expect(json).toHaveBeenCalledWith({ error: 'Failed to delete group' });
|
||||
});
|
||||
|
||||
it('returns 200 even when cascade cleanup partially fails', async () => {
|
||||
it('returns 500 and keeps the group when cascade cleanup fails', async () => {
|
||||
const deps = createDeps({
|
||||
deleteGroup: jest.fn().mockResolvedValue(mockGroup()),
|
||||
findGroupById: jest.fn().mockResolvedValue(mockGroup()),
|
||||
deleteAclEntries: jest.fn().mockRejectedValue(new Error('cleanup failed')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
|
|
@ -869,17 +874,358 @@ describe('createAdminGroupsHandlers', () => {
|
|||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(200);
|
||||
expect(json).toHaveBeenCalledWith({ success: true, id: validId });
|
||||
expect(deps.deleteConfig).toHaveBeenCalledWith(PrincipalType.GROUP, validId);
|
||||
expect(deps.deleteAclEntries).toHaveBeenCalledWith({
|
||||
principalType: PrincipalType.GROUP,
|
||||
principalId: new Types.ObjectId(validId),
|
||||
expect(status).toHaveBeenCalledWith(500);
|
||||
expect(json).toHaveBeenCalledWith(expect.objectContaining({ code: 'GROUP_CLEANUP_FAILED' }));
|
||||
expect(deps.deleteGroup).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
describe('compensation and audit', () => {
|
||||
const withUser = <T extends { req: ServerRequest }>(reqRes: T): T => {
|
||||
(reqRes.req as unknown as { user: Partial<IUser> }).user = {
|
||||
_id: new Types.ObjectId(),
|
||||
name: 'Admin',
|
||||
};
|
||||
return reqRes;
|
||||
};
|
||||
const config = { principalType: 'group', principalId: 'p', overrides: {} };
|
||||
|
||||
function compensationDeps(overrides: Partial<AdminGroupsDeps> = {}) {
|
||||
return createDeps({
|
||||
findGroupById: jest.fn().mockResolvedValue(mockGroup()),
|
||||
deleteConfig: jest.fn().mockResolvedValue(config),
|
||||
restoreConfig: jest.fn().mockResolvedValue(true),
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
|
||||
it('restores the config when the ACL removal fails after it', async () => {
|
||||
const deps = compensationDeps({
|
||||
deleteAclEntries: jest.fn().mockRejectedValue(new Error('acl down')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status, json } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(500);
|
||||
expect(json).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ code: 'GROUP_CLEANUP_FAILED' }),
|
||||
);
|
||||
expect(deps.restoreConfig).toHaveBeenCalledWith(config);
|
||||
expect(deps.deleteGroup).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not restore the config when a concurrent delete already removed the group', async () => {
|
||||
const deps = compensationDeps({
|
||||
findGroupById: jest.fn().mockResolvedValueOnce(mockGroup()).mockResolvedValueOnce(null),
|
||||
deleteAclEntries: jest.fn().mockRejectedValue(new Error('acl down')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(500);
|
||||
expect(deps.restoreConfig).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('restores the config when the group state recheck fails', async () => {
|
||||
const deps = compensationDeps({
|
||||
findGroupById: jest
|
||||
.fn()
|
||||
.mockResolvedValueOnce(mockGroup())
|
||||
.mockRejectedValueOnce(new Error('db down')),
|
||||
deleteAclEntries: jest.fn().mockRejectedValue(new Error('acl down')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(500);
|
||||
expect(deps.restoreConfig).toHaveBeenCalledWith(config);
|
||||
});
|
||||
|
||||
it('restores the config when the delete fails and the group state cannot be read', async () => {
|
||||
const deps = compensationDeps({
|
||||
findGroupById: jest
|
||||
.fn()
|
||||
.mockResolvedValueOnce(mockGroup())
|
||||
.mockRejectedValueOnce(new Error('db down')),
|
||||
deleteGroup: jest.fn().mockRejectedValue(new Error('delete failed')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(500);
|
||||
expect(deps.restoreConfig).toHaveBeenCalledWith(config);
|
||||
});
|
||||
|
||||
it('restores the config from a pre-delete snapshot when the removal rejects after committing', async () => {
|
||||
const deps = compensationDeps({
|
||||
findConfigByPrincipal: jest.fn().mockResolvedValue(config),
|
||||
deleteConfig: jest.fn().mockRejectedValue(new Error('ack lost')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(500);
|
||||
expect(deps.findConfigByPrincipal).toHaveBeenCalledWith('group', validId, {
|
||||
includeInactive: true,
|
||||
});
|
||||
expect(deps.restoreConfig).toHaveBeenCalledWith(config);
|
||||
expect(deps.deleteGroup).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('completes the deletion when the config restore itself fails', async () => {
|
||||
const deps = compensationDeps({
|
||||
restoreConfig: jest.fn().mockRejectedValue(new Error('restore down')),
|
||||
deleteAclEntries: jest.fn().mockRejectedValue(new Error('acl down')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(deps.deleteGroup).toHaveBeenCalledWith(validId);
|
||||
expect(deps.deleteAclEntries).toHaveBeenCalledTimes(2);
|
||||
expect(status).toHaveBeenCalledWith(200);
|
||||
});
|
||||
|
||||
it('treats a compensating deletion that throws after committing as completed', async () => {
|
||||
const group = mockGroup();
|
||||
const deps = compensationDeps({
|
||||
restoreConfig: jest.fn().mockRejectedValue(new Error('restore down')),
|
||||
deleteAclEntries: jest.fn().mockRejectedValue(new Error('acl down')),
|
||||
findGroupById: jest
|
||||
.fn()
|
||||
.mockResolvedValueOnce(group)
|
||||
.mockResolvedValueOnce(group)
|
||||
.mockResolvedValue(null),
|
||||
deleteGroup: jest.fn().mockRejectedValue(new Error('cache invalidation failed')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(200);
|
||||
});
|
||||
|
||||
it('reports not found when a concurrent delete won the race during the compensating deletion', async () => {
|
||||
const deps = compensationDeps({
|
||||
restoreConfig: jest.fn().mockRejectedValue(new Error('restore down')),
|
||||
deleteAclEntries: jest.fn().mockRejectedValue(new Error('acl down')),
|
||||
deleteGroup: jest.fn().mockResolvedValue(null),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(404);
|
||||
});
|
||||
|
||||
it('reports GROUP_RESTORE_FAILED when neither the restore nor the deletion succeeds', async () => {
|
||||
const deps = compensationDeps({
|
||||
restoreConfig: jest.fn().mockRejectedValue(new Error('restore down')),
|
||||
deleteAclEntries: jest.fn().mockRejectedValue(new Error('acl down')),
|
||||
deleteGroup: jest.fn().mockRejectedValue(new Error('delete down')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status, json } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(500);
|
||||
expect(json).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ code: 'GROUP_RESTORE_FAILED' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('restores the config but never the ACL entries when the group delete fails', async () => {
|
||||
const deps = compensationDeps({
|
||||
deleteGroup: jest.fn().mockRejectedValue(new Error('db down')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(500);
|
||||
expect(deps.deleteAclEntries).toHaveBeenCalled();
|
||||
expect(deps.restoreConfig).toHaveBeenCalledWith(config);
|
||||
});
|
||||
|
||||
it('leaves the removed ACL entries removed when the config removal fails', async () => {
|
||||
const deps = compensationDeps({
|
||||
deleteConfig: jest.fn().mockRejectedValue(new Error('config down')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(500);
|
||||
expect(deps.restoreConfig).not.toHaveBeenCalled();
|
||||
expect(deps.deleteGroup).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not restore dependents of a group that is already gone', async () => {
|
||||
const deps = compensationDeps({
|
||||
findGroupById: jest.fn().mockResolvedValueOnce(mockGroup()).mockResolvedValue(null),
|
||||
deleteGroup: jest.fn().mockRejectedValue(new Error('db down')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(200);
|
||||
expect(deps.restoreConfig).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('records a pending group.deleted before any removal and a failure entry after a failed cleanup', async () => {
|
||||
const order: string[] = [];
|
||||
const recordAuditEntry = jest.fn().mockImplementation(async (input) => {
|
||||
order.push(`audit:${input.outcome ?? 'success'}`);
|
||||
});
|
||||
const deps = compensationDeps({
|
||||
recordAuditEntry,
|
||||
deleteConfig: jest.fn().mockImplementation(async () => {
|
||||
order.push('config');
|
||||
return config;
|
||||
}),
|
||||
deleteAclEntries: jest.fn().mockRejectedValue(new Error('acl down')),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res } = withUser(createReqRes({ params: { id: validId } }));
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(order).toEqual(['audit:pending', 'config', 'audit:failure']);
|
||||
expect(recordAuditEntry).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'group.deleted',
|
||||
outcome: 'failure',
|
||||
metadata: { code: 'GROUP_CLEANUP_FAILED' },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('appends a not_found failure entry when the request loses the race', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const deps = compensationDeps({
|
||||
recordAuditEntry,
|
||||
deleteGroup: jest.fn().mockResolvedValue(null),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = withUser(createReqRes({ params: { id: validId } }));
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(404);
|
||||
expect(recordAuditEntry).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({ outcome: 'failure', metadata: { code: 'not_found' } }),
|
||||
);
|
||||
expect(deps.restoreConfig).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not wait for a slow pending audit write before removing dependents', async () => {
|
||||
let releaseAudit: () => void = () => undefined;
|
||||
const order: string[] = [];
|
||||
const recordAuditEntry = jest.fn().mockImplementation(async (input) => {
|
||||
if (input.outcome === 'pending') {
|
||||
await new Promise<void>((resolve) => (releaseAudit = resolve));
|
||||
}
|
||||
order.push(`audit:${input.outcome ?? 'success'}`);
|
||||
});
|
||||
const deps = compensationDeps({
|
||||
recordAuditEntry,
|
||||
deleteGroup: jest.fn().mockImplementation(async () => {
|
||||
order.push('group');
|
||||
releaseAudit();
|
||||
return mockGroup();
|
||||
}),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = withUser(createReqRes({ params: { id: validId } }));
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(200);
|
||||
expect(order).toEqual(['group', 'audit:pending', 'audit:success']);
|
||||
});
|
||||
|
||||
it('still appends the terminal outcome when member-removal audits fail under fail-closed', async () => {
|
||||
const recordAuditEntry = jest.fn().mockImplementation(async (input) => {
|
||||
if (input.action === 'group.member_removed') {
|
||||
throw new Error('audit down');
|
||||
}
|
||||
});
|
||||
const group = mockGroup({ memberIds: ['m1'] });
|
||||
const deps = compensationDeps({
|
||||
recordAuditEntry,
|
||||
auditFailClosed: true,
|
||||
findGroupById: jest.fn().mockResolvedValue(group),
|
||||
deleteGroup: jest.fn().mockResolvedValue(group),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = withUser(createReqRes({ params: { id: validId } }));
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(200);
|
||||
expect(recordAuditEntry).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({ action: 'group.deleted', outcome: 'success' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('never blocks the delete on an audit outage, even when fail-closed', async () => {
|
||||
const deps = compensationDeps({
|
||||
recordAuditEntry: jest.fn().mockRejectedValue(new Error('audit down')),
|
||||
auditFailClosed: true,
|
||||
deleteGroup: jest.fn().mockResolvedValue(mockGroup()),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = withUser(createReqRes({ params: { id: validId } }));
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(200);
|
||||
expect(deps.deleteGroup).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
it('removes dependents before the group', async () => {
|
||||
const order: string[] = [];
|
||||
const deps = createDeps({
|
||||
findGroupById: jest.fn().mockResolvedValue(mockGroup()),
|
||||
deleteConfig: jest.fn().mockImplementation(async () => void order.push('config')),
|
||||
deleteAclEntries: jest.fn().mockImplementation(async () => void order.push('acl')),
|
||||
deleteGroup: jest.fn().mockImplementation(async () => {
|
||||
order.push('group');
|
||||
return mockGroup();
|
||||
}),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res } = createReqRes({ params: { id: validId } });
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(order[order.length - 1]).toBe('group');
|
||||
expect(order).toHaveLength(3);
|
||||
});
|
||||
|
||||
it('cleans up Config and AclEntry on group delete', async () => {
|
||||
const deps = createDeps({ deleteGroup: jest.fn().mockResolvedValue(mockGroup()) });
|
||||
const deps = createDeps({
|
||||
findGroupById: jest.fn().mockResolvedValue(mockGroup()),
|
||||
deleteGroup: jest.fn().mockResolvedValue(mockGroup()),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = createReqRes({ params: { id: validId } });
|
||||
|
||||
|
|
@ -1521,16 +1867,20 @@ describe('createAdminGroupsHandlers', () => {
|
|||
|
||||
it('records group.deleted', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const deps = createDeps({ recordAuditEntry });
|
||||
const deps = createDeps({
|
||||
recordAuditEntry,
|
||||
findGroupById: jest.fn().mockResolvedValue(mockGroup()),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
const { req, res, status } = withUser(createReqRes({ params: { id: validId } }));
|
||||
|
||||
await handlers.deleteGroup(req, res);
|
||||
|
||||
expect(status).toHaveBeenCalledWith(200);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect(recordAuditEntry).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'group.deleted',
|
||||
outcome: 'success',
|
||||
target: { type: 'group', id: validId, name: 'Test Group' },
|
||||
}),
|
||||
);
|
||||
|
|
@ -1540,6 +1890,7 @@ describe('createAdminGroupsHandlers', () => {
|
|||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const deps = createDeps({
|
||||
recordAuditEntry,
|
||||
findGroupById: jest.fn().mockResolvedValue(mockGroup({ memberIds: ['m1', 'm2'] })),
|
||||
deleteGroup: jest.fn().mockResolvedValue(mockGroup({ memberIds: ['m1', 'm2'] })),
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
|
|
@ -1629,7 +1980,7 @@ describe('createAdminGroupsHandlers', () => {
|
|||
it('does not record when the group is missing', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const deps = createDeps({
|
||||
deleteGroup: jest.fn().mockResolvedValue(null),
|
||||
findGroupById: jest.fn().mockResolvedValue(null),
|
||||
recordAuditEntry,
|
||||
});
|
||||
const handlers = createAdminGroupsHandlers(deps);
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ import type { ValidationError } from '~/types/error';
|
|||
import type { ServerRequest } from '~/types/http';
|
||||
import type { RecordAuditEntry } from './audit';
|
||||
import { getSafeErrorMetadata } from '~/utils/errors';
|
||||
import { cascadeStep, runCascade } from './cascade';
|
||||
import { createAdminAuditEmitter } from './audit';
|
||||
import { parsePagination } from './pagination';
|
||||
|
||||
|
|
@ -88,6 +89,16 @@ export interface AdminGroupsDeps {
|
|||
principalType: PrincipalType;
|
||||
principalId: string | Types.ObjectId;
|
||||
}) => Promise<DeleteResult>;
|
||||
/** Re-creates a removed config override in full, only if none exists, when the group
|
||||
* deletion cannot finish. ACL entries are never restored. Resolves to whether it inserted. */
|
||||
restoreConfig?: (snapshot: Record<string, unknown>) => Promise<boolean>;
|
||||
/** Reads the override before it is removed, so a rejected removal whose commit was
|
||||
* lost in transit can still be put back. */
|
||||
findConfigByPrincipal?: (
|
||||
principalType: PrincipalType,
|
||||
principalId: string,
|
||||
options?: { includeInactive?: boolean },
|
||||
) => Promise<object | null>;
|
||||
/** Optional audit emission for group mutations; fail-open unless `auditFailClosed`. */
|
||||
recordAuditEntry?: RecordAuditEntry;
|
||||
/** When true, a failed audit write surfaces as a 5xx after the mutation has
|
||||
|
|
@ -122,10 +133,16 @@ export function createAdminGroupsHandlers(deps: AdminGroupsDeps): {
|
|||
findUsers,
|
||||
deleteConfig,
|
||||
deleteAclEntries,
|
||||
restoreConfig,
|
||||
findConfigByPrincipal,
|
||||
recordAuditEntry,
|
||||
auditFailClosed,
|
||||
} = deps;
|
||||
|
||||
const emitRevocationAudit = createAdminAuditEmitter({
|
||||
recordAuditEntry,
|
||||
logPrefix: '[adminGroups]',
|
||||
});
|
||||
const emitAudit = createAdminAuditEmitter({
|
||||
recordAuditEntry,
|
||||
failClosed: auditFailClosed,
|
||||
|
|
@ -355,44 +372,231 @@ export function createAdminGroupsHandlers(deps: AdminGroupsDeps): {
|
|||
if (!isValidObjectIdString(id)) {
|
||||
return res.status(400).json({ error: 'Invalid group ID format' });
|
||||
}
|
||||
const deleted = await deleteGroup(id);
|
||||
if (!deleted) {
|
||||
const existing = await findGroupById(id);
|
||||
if (!existing) {
|
||||
return res.status(404).json({ error: 'Group not found' });
|
||||
}
|
||||
const target = groupTarget(existing, id);
|
||||
/**
|
||||
* The attempt is recorded as `pending` before any removal so a crash mid-cascade
|
||||
* still leaves a trail, but the write is started without being awaited: deleting a
|
||||
* group revokes access, so audit latency must not delay the cleanup. Every
|
||||
* later entry waits for it to settle to keep the chain ordered.
|
||||
* A `success` or `failure` entry is appended once the outcome is known.
|
||||
*/
|
||||
const attempt = emitRevocationAudit(req, {
|
||||
action: 'group.deleted',
|
||||
outcome: 'pending',
|
||||
severity: 'warning',
|
||||
target,
|
||||
});
|
||||
const record = async (outcome: 'success' | 'failure', code?: string) => {
|
||||
await attempt;
|
||||
return emitRevocationAudit(req, {
|
||||
action: 'group.deleted',
|
||||
outcome,
|
||||
severity: 'warning',
|
||||
target,
|
||||
...(code ? { metadata: { code } } : {}),
|
||||
});
|
||||
};
|
||||
const recordFailure = (code: string) => record('failure', code);
|
||||
/**
|
||||
* Config overrides and ACL entries are keyed by group id, so they are
|
||||
* removed before the group itself: if any removal fails the group stays
|
||||
* and the admin can retry, instead of leaving orphaned entries that a later
|
||||
* principal could inherit. Only the config override is put back (insert-only,
|
||||
* so one written since is kept): removing it can widen what members may do,
|
||||
* while ACL entries stay removed because that only narrows access and
|
||||
* re-inserting them could resurrect a grant revoked in the meantime.
|
||||
* deleteAclEntries is a raw deleteMany wrapper with no type casting.
|
||||
* grantPermission stores group principalId as ObjectId, so we must
|
||||
* cast here. deleteConfig normalizes internally.
|
||||
*/
|
||||
const cleanupResults = await Promise.allSettled([
|
||||
deleteConfig(PrincipalType.GROUP, id),
|
||||
deleteAclEntries({
|
||||
let restoreFailed = false;
|
||||
const cascade = await runCascade(
|
||||
[
|
||||
cascadeStep(
|
||||
'config',
|
||||
() => deleteConfig(PrincipalType.GROUP, id),
|
||||
restoreConfig
|
||||
? async (config) => {
|
||||
if (!config) {
|
||||
return;
|
||||
}
|
||||
const snapshot = (config as { toObject?: () => unknown }).toObject?.() ?? config;
|
||||
let restored: boolean;
|
||||
try {
|
||||
restored = await restoreConfig(snapshot as Record<string, unknown>);
|
||||
restoreFailed = false;
|
||||
} catch (error) {
|
||||
restoreFailed = true;
|
||||
throw error;
|
||||
}
|
||||
if (!restored) {
|
||||
logger.warn('[adminGroups] config restore skipped, override exists:', id);
|
||||
}
|
||||
}
|
||||
: undefined,
|
||||
findConfigByPrincipal
|
||||
? () => findConfigByPrincipal(PrincipalType.GROUP, id, { includeInactive: true })
|
||||
: undefined,
|
||||
),
|
||||
cascadeStep('acl', () =>
|
||||
deleteAclEntries({
|
||||
principalType: PrincipalType.GROUP,
|
||||
principalId: new Types.ObjectId(id),
|
||||
}),
|
||||
),
|
||||
],
|
||||
'[adminGroups]',
|
||||
);
|
||||
/**
|
||||
* Puts the config override back. When that restore itself fails the group would live on
|
||||
* without its restriction, so the deletion is completed instead of retaining it.
|
||||
*/
|
||||
const compensate = async (): Promise<'restored' | 'stuck' | 'missing' | IGroup> => {
|
||||
await cascade.compensate();
|
||||
if (!restoreFailed) {
|
||||
return 'restored';
|
||||
}
|
||||
logger.error('[adminGroups] config restore failed, completing the deletion:', id);
|
||||
let removed: IGroup | null = null;
|
||||
try {
|
||||
removed = await deleteGroup(id);
|
||||
if (!removed) {
|
||||
/** Another request deleted the group first; this one removed nothing. */
|
||||
return 'missing';
|
||||
}
|
||||
} catch (error) {
|
||||
/** The delete may have committed before it threw (post-delete cache work); a confirmed
|
||||
* absence is a completed deletion. */
|
||||
const absent = await findGroupById(id).then(
|
||||
(group) => group == null,
|
||||
() => false,
|
||||
);
|
||||
if (!absent) {
|
||||
logger.error(
|
||||
'[adminGroups] group left without its config override:',
|
||||
id,
|
||||
getSafeErrorMetadata(error),
|
||||
);
|
||||
return 'stuck';
|
||||
}
|
||||
removed = existing;
|
||||
}
|
||||
/** The group is gone for good, so a leftover ACL entry is orphaned: retry its removal once. */
|
||||
await deleteAclEntries({
|
||||
principalType: PrincipalType.GROUP,
|
||||
principalId: new Types.ObjectId(id),
|
||||
}),
|
||||
]);
|
||||
for (const result of cleanupResults) {
|
||||
if (result.status === 'rejected') {
|
||||
logger.error('[adminGroups] cascade cleanup step failed for group:', id, result.reason);
|
||||
}
|
||||
}
|
||||
}).catch((aclError) =>
|
||||
logger.error(
|
||||
'[adminGroups] acl cleanup retry failed after group deletion:',
|
||||
getSafeErrorMetadata(aclError),
|
||||
),
|
||||
);
|
||||
return removed;
|
||||
};
|
||||
/** Every member loses group-scoped access, so the affected accounts are recorded (chunked). */
|
||||
const revoked = [...new Set(deleted.memberIds ?? [])];
|
||||
for (let i = 0; i < revoked.length; i += AUDIT_MEMBER_CHUNK) {
|
||||
const chunk = revoked.slice(i, i + AUDIT_MEMBER_CHUNK);
|
||||
await emitAudit(req, {
|
||||
action: 'group.member_removed',
|
||||
severity: 'warning',
|
||||
target: groupTarget(deleted, id),
|
||||
metadata: { userIds: chunk.join(','), count: chunk.length, cause: 'group_deleted' },
|
||||
const recordMemberRemovals = async (group: IGroup) => {
|
||||
/** Keeps the chain causal: the pending attempt precedes the member entries. */
|
||||
await attempt;
|
||||
const revoked = [...new Set(group.memberIds ?? [])];
|
||||
for (let i = 0; i < revoked.length; i += AUDIT_MEMBER_CHUNK) {
|
||||
const chunk = revoked.slice(i, i + AUDIT_MEMBER_CHUNK);
|
||||
await emitRevocationAudit(req, {
|
||||
action: 'group.member_removed',
|
||||
severity: 'warning',
|
||||
target: groupTarget(group, id),
|
||||
metadata: { userIds: chunk.join(','), count: chunk.length, cause: 'group_deleted' },
|
||||
});
|
||||
}
|
||||
};
|
||||
const compensatedResponse = async (outcome: 'stuck' | 'missing' | IGroup) => {
|
||||
if (outcome === 'missing') {
|
||||
await recordFailure('not_found');
|
||||
return res.status(404).json({ error: 'Group not found' });
|
||||
}
|
||||
if (outcome !== 'stuck') {
|
||||
await recordMemberRemovals(outcome);
|
||||
res.status(200).json({ success: true, id });
|
||||
await record('success');
|
||||
return res;
|
||||
}
|
||||
await recordFailure('GROUP_RESTORE_FAILED');
|
||||
return res.status(500).json({
|
||||
error: 'Failed to delete group and its config override could not be restored',
|
||||
code: 'GROUP_RESTORE_FAILED',
|
||||
});
|
||||
};
|
||||
const cleanupFailedResponse = async () => {
|
||||
await recordFailure('GROUP_CLEANUP_FAILED');
|
||||
return res.status(500).json({
|
||||
error: 'Failed to remove the group permissions and config; the group was not deleted',
|
||||
code: 'GROUP_CLEANUP_FAILED',
|
||||
});
|
||||
};
|
||||
|
||||
if (cascade.failed.length > 0) {
|
||||
/** A concurrent delete may have removed the group meanwhile; restoring its config would
|
||||
* orphan it. When the lookup itself fails the group state is unknown, and putting the
|
||||
* override back is the safe side: a leftover override only restricts, a lost one widens. */
|
||||
const gone = await findGroupById(id).then(
|
||||
(group) => group == null,
|
||||
() => false,
|
||||
);
|
||||
if (!gone) {
|
||||
const outcome = await compensate();
|
||||
if (outcome !== 'restored') {
|
||||
return compensatedResponse(outcome);
|
||||
}
|
||||
}
|
||||
return cleanupFailedResponse();
|
||||
}
|
||||
await emitAudit(req, {
|
||||
action: 'group.deleted',
|
||||
severity: 'warning',
|
||||
target: groupTarget(deleted, id),
|
||||
});
|
||||
return res.status(200).json({ success: true, id });
|
||||
|
||||
let deleted: IGroup | null;
|
||||
try {
|
||||
deleted = await deleteGroup(id);
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
'[adminGroups] deleteGroup failed after cascade cleanup:',
|
||||
getSafeErrorMetadata(error),
|
||||
);
|
||||
/** The delete may have committed before it threw. Restoring dependents for a
|
||||
* group that is gone would orphan them. */
|
||||
const lookup = await findGroupById(id).then(
|
||||
(group) => ({ known: true, group }),
|
||||
() => ({ known: false, group: null }),
|
||||
);
|
||||
if (lookup.known && lookup.group) {
|
||||
const outcome = await compensate();
|
||||
if (outcome !== 'restored') {
|
||||
return compensatedResponse(outcome);
|
||||
}
|
||||
await recordFailure('GROUP_CLEANUP_FAILED');
|
||||
return res.status(500).json({ error: 'Failed to delete group' });
|
||||
}
|
||||
if (!lookup.known) {
|
||||
/** Unknown state favors the restriction: a leftover override narrows access, a lost one widens it. */
|
||||
const outcome = await compensate();
|
||||
if (outcome !== 'restored') {
|
||||
return compensatedResponse(outcome);
|
||||
}
|
||||
logger.error('[adminGroups] group state unverifiable after delete failure:', id);
|
||||
await recordFailure('GROUP_STATE_UNVERIFIABLE');
|
||||
return res.status(500).json({ error: 'Failed to delete group' });
|
||||
}
|
||||
/** Confirmed absent: the deletion did commit. */
|
||||
deleted = existing;
|
||||
}
|
||||
if (!deleted) {
|
||||
await recordFailure('not_found');
|
||||
return res.status(404).json({ error: 'Group not found' });
|
||||
}
|
||||
await recordMemberRemovals(deleted);
|
||||
res.status(200).json({ success: true, id });
|
||||
await record('success');
|
||||
return res;
|
||||
} catch (error) {
|
||||
logger.error('[adminGroups] deleteGroup error:', getSafeErrorMetadata(error));
|
||||
return res.status(500).json({ error: 'Failed to delete group' });
|
||||
|
|
|
|||
|
|
@ -547,6 +547,69 @@ describe('createAdminLangfuseHandlers', () => {
|
|||
expect(JSON.stringify(deps.recordConnectionUpdate.mock.calls)).not.toContain('pk-lf-1');
|
||||
});
|
||||
|
||||
it('records langfuse.updated with the changes and never the keys', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const { handlers } = createHandlers({ recordAuditEntry });
|
||||
const res = mockRes();
|
||||
|
||||
await handlers.updateConnection(
|
||||
mockReq({
|
||||
body: {
|
||||
enabled: true,
|
||||
destination: 'eu',
|
||||
publicKey: 'pk-lf-1',
|
||||
secretKey: 'sk-lf-secret',
|
||||
},
|
||||
}),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(res.statusCode).toBe(200);
|
||||
expect(recordAuditEntry).toHaveBeenCalledTimes(1);
|
||||
const input = recordAuditEntry.mock.calls[0][0];
|
||||
expect(input).toEqual(
|
||||
expect.objectContaining({
|
||||
action: 'langfuse.updated',
|
||||
target: { type: 'langfuse', id: 'connection' },
|
||||
metadata: { changes: 'created', enabled: true, destination: 'eu' },
|
||||
}),
|
||||
);
|
||||
expect(JSON.stringify(input)).not.toContain('sk-lf-secret');
|
||||
expect(JSON.stringify(input)).not.toContain('pk-lf-1');
|
||||
});
|
||||
|
||||
it('starts cache invalidation before waiting on the audit write', async () => {
|
||||
const order: string[] = [];
|
||||
const recordAuditEntry = jest.fn().mockImplementation(async () => {
|
||||
order.push('audit');
|
||||
});
|
||||
const invalidateConfigCaches = jest.fn().mockImplementation(async () => {
|
||||
order.push('invalidate');
|
||||
});
|
||||
const { handlers } = createHandlers({ recordAuditEntry, invalidateConfigCaches });
|
||||
|
||||
await handlers.updateConnection(
|
||||
mockReq({
|
||||
body: { enabled: true, destination: 'eu', publicKey: 'pk-lf-1', secretKey: 'sk-lf-2' },
|
||||
}),
|
||||
mockRes(),
|
||||
);
|
||||
|
||||
expect(order[0]).toBe('invalidate');
|
||||
expect(order).toContain('audit');
|
||||
});
|
||||
|
||||
it('records nothing when the update is rejected', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const { handlers } = createHandlers({ recordAuditEntry });
|
||||
const res = mockRes();
|
||||
|
||||
await handlers.updateConnection(mockReq({ body: { enabled: true } }), res);
|
||||
|
||||
expect(res.statusCode).toBe(400);
|
||||
expect(recordAuditEntry).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('requires a new secret when connection fields change', async () => {
|
||||
const { handlers, deps } = createHandlers({
|
||||
findConfigByPrincipal: jest
|
||||
|
|
@ -771,6 +834,92 @@ describe('createAdminLangfuseHandlers', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('updateConnection activation failure', () => {
|
||||
it('still audits and invalidates caches when activation fails after the patch committed', async () => {
|
||||
const inactive = {
|
||||
...baseConfigDoc({
|
||||
enabled: true,
|
||||
destination: 'eu',
|
||||
publicKey: 'pk-lf-1',
|
||||
secretKey: encryptV3('sk-lf-secret'),
|
||||
}),
|
||||
isActive: false,
|
||||
};
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const invalidateConfigCaches = jest.fn().mockResolvedValue(undefined);
|
||||
const { handlers } = createHandlers({
|
||||
findConfigByPrincipal: jest.fn().mockResolvedValue(inactive),
|
||||
patchConfigFields: jest.fn().mockResolvedValue(inactive),
|
||||
toggleConfigActive: jest.fn().mockRejectedValue(new Error('toggle failed')),
|
||||
recordAuditEntry,
|
||||
invalidateConfigCaches,
|
||||
});
|
||||
const res = mockRes();
|
||||
|
||||
await handlers.updateConnection(
|
||||
mockReq({
|
||||
body: {
|
||||
enabled: true,
|
||||
destination: 'eu',
|
||||
publicKey: 'pk-lf-1',
|
||||
secretKey: 'sk-lf-secret',
|
||||
},
|
||||
}),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(res.statusCode).toBe(500);
|
||||
expect(invalidateConfigCaches).toHaveBeenCalled();
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'langfuse.updated',
|
||||
metadata: expect.objectContaining({ enabled: false, activation: 'failed' }),
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('updateConnection activation returning null', () => {
|
||||
it('treats a vanished config as a failed activation', async () => {
|
||||
const inactive = {
|
||||
...baseConfigDoc({
|
||||
enabled: true,
|
||||
destination: 'eu',
|
||||
publicKey: 'pk-lf-1',
|
||||
secretKey: encryptV3('sk-lf-secret'),
|
||||
}),
|
||||
isActive: false,
|
||||
};
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const { handlers } = createHandlers({
|
||||
findConfigByPrincipal: jest.fn().mockResolvedValue(inactive),
|
||||
patchConfigFields: jest.fn().mockResolvedValue(inactive),
|
||||
toggleConfigActive: jest.fn().mockResolvedValue(null),
|
||||
recordAuditEntry,
|
||||
});
|
||||
const res = mockRes();
|
||||
|
||||
await handlers.updateConnection(
|
||||
mockReq({
|
||||
body: {
|
||||
enabled: true,
|
||||
destination: 'eu',
|
||||
publicKey: 'pk-lf-1',
|
||||
secretKey: 'sk-lf-secret',
|
||||
},
|
||||
}),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(res.statusCode).toBe(500);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
metadata: expect.objectContaining({ enabled: false, activation: 'failed' }),
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('testConnection', () => {
|
||||
it('requires destination and publicKey', async () => {
|
||||
const { handlers } = createHandlers();
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ import type { Types, ClientSession } from 'mongoose';
|
|||
import type { Response } from 'express';
|
||||
import type { LangfuseTenantDestination } from '~/langfuse/tenantDestinations';
|
||||
import type { ServerRequest } from '~/types/http';
|
||||
import type { RecordAuditEntry } from './audit';
|
||||
import {
|
||||
getLangfuseTenantDestinations,
|
||||
resolveLangfuseTenantDestination,
|
||||
|
|
@ -23,6 +24,7 @@ import { decryptConfigSecret, encryptConfigSecretFields } from './secrets';
|
|||
import { scopeHeadersToDestination } from '~/langfuse/destinations';
|
||||
import { isLangfuseConnectionAvailable } from '~/langfuse/policy';
|
||||
import { resolveLangfuseSession } from '~/langfuse/session';
|
||||
import { createAdminAuditEmitter } from './audit';
|
||||
import { mergeHeaders } from '~/utils/headers';
|
||||
|
||||
const DEFAULT_PRIORITY = 10;
|
||||
|
|
@ -46,7 +48,7 @@ export interface LangfuseConnectionEvent {
|
|||
destination?: string;
|
||||
change: LangfuseConnectionChange;
|
||||
changes: LangfuseConnectionChange[];
|
||||
verification_result: 'skipped' | 'success';
|
||||
verification_result: 'skipped' | 'success' | 'failed';
|
||||
}
|
||||
|
||||
export interface AdminLangfuseDeps {
|
||||
|
|
@ -73,6 +75,9 @@ export interface AdminLangfuseDeps {
|
|||
getMessages: MessageMethods['getMessages'];
|
||||
invalidateConfigCaches?: (tenantId?: string) => Promise<void>;
|
||||
recordConnectionUpdate?: (event: LangfuseConnectionEvent) => void;
|
||||
/** Optional audit emission for connection updates; fail-open unless `auditFailClosed`. */
|
||||
recordAuditEntry?: RecordAuditEntry;
|
||||
auditFailClosed?: boolean;
|
||||
}
|
||||
|
||||
function getTenantId(req: ServerRequest): string | undefined {
|
||||
|
|
@ -294,7 +299,14 @@ export function createAdminLangfuseHandlers(deps: AdminLangfuseDeps): {
|
|||
invalidateConfigCaches,
|
||||
recordConnectionUpdate = (event) =>
|
||||
logger.info({ message: '[adminLangfuse] Connection updated', ...event }),
|
||||
recordAuditEntry,
|
||||
auditFailClosed,
|
||||
} = deps;
|
||||
const emitAudit = createAdminAuditEmitter({
|
||||
recordAuditEntry,
|
||||
failClosed: auditFailClosed,
|
||||
logPrefix: '[adminLangfuse]',
|
||||
});
|
||||
|
||||
function findBaseConfig(options?: { includeInactive?: boolean }): Promise<IConfig | null> {
|
||||
return options
|
||||
|
|
@ -443,8 +455,25 @@ export function createAdminLangfuseHandlers(deps: AdminLangfuseDeps): {
|
|||
encryptConfigSecretFields(fields),
|
||||
existing?.priority ?? DEFAULT_PRIORITY,
|
||||
);
|
||||
/** The patch has committed by now, so a failed activation must not skip the cache
|
||||
* invalidation or the audit entry; it is rethrown once both have started. */
|
||||
let activationError: unknown;
|
||||
if (updated?.isActive === false) {
|
||||
updated = await toggleConfigActive(PrincipalType.ROLE, BASE_CONFIG_PRINCIPAL_ID, true);
|
||||
try {
|
||||
const activated = await toggleConfigActive(
|
||||
PrincipalType.ROLE,
|
||||
BASE_CONFIG_PRINCIPAL_ID,
|
||||
true,
|
||||
);
|
||||
if (activated) {
|
||||
updated = activated;
|
||||
} else {
|
||||
/** A null result means the config vanished meanwhile (a concurrent delete). */
|
||||
activationError = new Error('Langfuse base config no longer exists');
|
||||
}
|
||||
} catch (error) {
|
||||
activationError = error;
|
||||
}
|
||||
}
|
||||
|
||||
const status = buildStatus(updated ?? existing);
|
||||
|
|
@ -455,21 +484,42 @@ export function createAdminLangfuseHandlers(deps: AdminLangfuseDeps): {
|
|||
publicKey,
|
||||
secretKey,
|
||||
);
|
||||
let verificationResult: LangfuseConnectionEvent['verification_result'] = 'skipped';
|
||||
if (activationError) {
|
||||
verificationResult = 'failed';
|
||||
} else if (connectionChanged) {
|
||||
verificationResult = 'success';
|
||||
}
|
||||
recordConnectionUpdate({
|
||||
event_name: 'librechat.langfuse.connection.changed',
|
||||
tenant_id: getTenantId(req),
|
||||
configured: status.configured,
|
||||
enabled: status.enabled,
|
||||
enabled: activationError ? false : status.enabled,
|
||||
destination: status.destination,
|
||||
change: changes[0],
|
||||
changes,
|
||||
verification_result: connectionChanged ? 'success' : 'skipped',
|
||||
verification_result: verificationResult,
|
||||
});
|
||||
|
||||
invalidateConfigCaches?.(getTenantId(req))?.catch((err) =>
|
||||
logger.error('[adminLangfuse] Cache invalidation failed after update:', err),
|
||||
);
|
||||
|
||||
/** Recorded after the cache invalidation starts so audit latency never delays it. Records which parts changed, never the keys themselves. */
|
||||
await emitAudit(req, {
|
||||
action: 'langfuse.updated',
|
||||
severity: 'warning',
|
||||
target: { type: 'langfuse', id: 'connection' },
|
||||
metadata: {
|
||||
changes: [...changes].sort().join(','),
|
||||
enabled: activationError ? false : status.enabled,
|
||||
...(activationError ? { activation: 'failed' } : {}),
|
||||
...(status.destination ? { destination: status.destination } : {}),
|
||||
},
|
||||
});
|
||||
|
||||
if (activationError) {
|
||||
throw activationError;
|
||||
}
|
||||
return res.status(200).json(status);
|
||||
} catch (error) {
|
||||
logger.error('[adminLangfuse] updateConnection error:', error);
|
||||
|
|
|
|||
|
|
@ -576,3 +576,67 @@ describe('createAdminSkillsSyncAccess', () => {
|
|||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('skill sync credential audit entries', () => {
|
||||
function build(overrides: Record<string, jest.Mock> = {}) {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const handlers = createAdminSkillsSyncHandlers({
|
||||
runner: { getStatus: jest.fn(), runOnce: jest.fn() } as never,
|
||||
upsertCredential: jest.fn().mockResolvedValue({
|
||||
provider: 'github',
|
||||
credentialKey: 'github-skills-prod',
|
||||
credentialPresent: true,
|
||||
tokenFingerprint: 'abcd',
|
||||
}),
|
||||
deleteCredential: jest.fn().mockResolvedValue({ deleted: true }),
|
||||
recordAuditEntry,
|
||||
...overrides,
|
||||
});
|
||||
return { handlers, recordAuditEntry };
|
||||
}
|
||||
const req = (extra: Record<string, unknown> = {}) =>
|
||||
({
|
||||
user: { id: 'admin-1', name: 'Admin', tenantId: 'tenant-a' },
|
||||
params: { credentialKey: 'github-skills-prod' },
|
||||
body: { token: 'ghp_supersecrettoken' },
|
||||
headers: {},
|
||||
...extra,
|
||||
}) as never;
|
||||
|
||||
it('records skill_credential.set on the platform chain without the token', async () => {
|
||||
const { handlers, recordAuditEntry } = build();
|
||||
|
||||
await handlers.setCredential(req(), createResponse());
|
||||
|
||||
expect(recordAuditEntry).toHaveBeenCalledTimes(1);
|
||||
const input = recordAuditEntry.mock.calls[0][0];
|
||||
expect(input).toEqual(
|
||||
expect.objectContaining({
|
||||
action: 'skill_credential.set',
|
||||
target: { type: 'skill_credential', id: 'github-skills-prod', name: 'github' },
|
||||
tenantId: undefined,
|
||||
}),
|
||||
);
|
||||
expect(JSON.stringify(input)).not.toContain('ghp_supersecrettoken');
|
||||
});
|
||||
|
||||
it('records skill_credential.deleted only when something was deleted', async () => {
|
||||
const missing = build({ deleteCredential: jest.fn().mockResolvedValue({ deleted: false }) });
|
||||
await missing.handlers.deleteCredential(req(), createResponse());
|
||||
expect(missing.recordAuditEntry).not.toHaveBeenCalled();
|
||||
|
||||
const { handlers, recordAuditEntry } = build();
|
||||
await handlers.deleteCredential(req(), createResponse());
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ action: 'skill_credential.deleted' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('records nothing for an invalid credential key', async () => {
|
||||
const { handlers, recordAuditEntry } = build();
|
||||
|
||||
await handlers.setCredential(req({ params: { credentialKey: 'bad key!' } }), createResponse());
|
||||
|
||||
expect(recordAuditEntry).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -17,6 +17,9 @@ import type {
|
|||
import type { NextFunction, Request, RequestHandler, Response } from 'express';
|
||||
import type { Types } from 'mongoose';
|
||||
import type { GitHubSkillSyncRunner } from '~/skills/sync';
|
||||
import type { ServerRequest } from '~/types/http';
|
||||
import type { RecordAuditEntry } from './audit';
|
||||
import { createAdminAuditEmitter } from './audit';
|
||||
|
||||
export type AdminSkillsRequest = Request & {
|
||||
user?: {
|
||||
|
|
@ -61,6 +64,9 @@ export type AdminSkillSyncDeps = {
|
|||
provider: SkillSyncProvider,
|
||||
credentialKey: string,
|
||||
) => Promise<{ deleted: boolean }>;
|
||||
/** Optional audit emission for credential changes; fail-open unless `auditFailClosed`. */
|
||||
recordAuditEntry?: RecordAuditEntry;
|
||||
auditFailClosed?: boolean;
|
||||
};
|
||||
|
||||
export type AdminSkillSyncAccessDeps = {
|
||||
|
|
@ -359,6 +365,12 @@ export function createAdminSkillsSyncAccess(deps: AdminSkillSyncAccessDeps): Adm
|
|||
}
|
||||
|
||||
export function createAdminSkillsSyncHandlers(deps: AdminSkillSyncDeps): AdminSkillsSyncHandlers {
|
||||
const emitAudit = createAdminAuditEmitter({
|
||||
recordAuditEntry: deps.recordAuditEntry,
|
||||
failClosed: deps.auditFailClosed,
|
||||
logPrefix: '[adminSkills]',
|
||||
});
|
||||
|
||||
function getRunner(req: Request): GitHubSkillSyncRunner {
|
||||
const runner = deps.getRunner?.(req) ?? deps.runner;
|
||||
if (!runner) {
|
||||
|
|
@ -411,6 +423,13 @@ export function createAdminSkillsSyncHandlers(deps: AdminSkillSyncDeps): AdminSk
|
|||
token: token.trim(),
|
||||
userId: getUserObjectId(req),
|
||||
});
|
||||
/** Credentials are platform-level, so entries go to the platform chain. */
|
||||
await emitAudit(req as unknown as ServerRequest, {
|
||||
action: 'skill_credential.set',
|
||||
severity: 'warning',
|
||||
target: { type: 'skill_credential', id: credentialKey, name: 'github' },
|
||||
tenantId: undefined,
|
||||
});
|
||||
return res.status(200).json(serializeCredential(credential));
|
||||
}
|
||||
|
||||
|
|
@ -420,6 +439,14 @@ export function createAdminSkillsSyncHandlers(deps: AdminSkillSyncDeps): AdminSk
|
|||
return res.status(400).json({ error: 'Invalid credential key' });
|
||||
}
|
||||
const result = await deps.deleteCredential('github', credentialKey);
|
||||
if (result.deleted) {
|
||||
await emitAudit(req as unknown as ServerRequest, {
|
||||
action: 'skill_credential.deleted',
|
||||
severity: 'warning',
|
||||
target: { type: 'skill_credential', id: credentialKey, name: 'github' },
|
||||
tenantId: undefined,
|
||||
});
|
||||
}
|
||||
return res.status(200).json({ credentialKey, deleted: result.deleted });
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -397,6 +397,33 @@ describe('passkey registration provider enforcement', () => {
|
|||
expect(await methods.countPasskeysByUser(user._id.toString())).toBe(0);
|
||||
});
|
||||
|
||||
it('audits a credential that survives a failed rollback', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const countPasskeysByUser = jest
|
||||
.fn(methods.countPasskeysByUser)
|
||||
.mockResolvedValueOnce(0)
|
||||
.mockRejectedValueOnce(new Error('Count unavailable'));
|
||||
const deletePasskey = jest.fn().mockRejectedValue(new Error('Delete unavailable'));
|
||||
const handlers = createPasskeyHandlers(
|
||||
buildDeps({ countPasskeysByUser, deletePasskey, recordAuditEntry }),
|
||||
);
|
||||
const user = await createUser();
|
||||
const res = buildRes();
|
||||
|
||||
await handlers.registerPasskeyVerify(
|
||||
authedReq(user, { credential: attestation, password: PASSWORD }),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(res.status).toHaveBeenCalledWith(500);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'passkey.registered',
|
||||
metadata: expect.objectContaining({ rollback: 'failed' }),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('rolls back a credential that wins the enrollment race past the cap', async () => {
|
||||
/** Both verifies observe the pre-enrollment count, as two concurrent
|
||||
* ceremonies for the same account do, so both pass the early check. */
|
||||
|
|
@ -474,6 +501,45 @@ describe('passkey registration provider enforcement', () => {
|
|||
});
|
||||
});
|
||||
|
||||
it('records passkey.registered with the user as actor', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const handlers = createPasskeyHandlers(buildDeps({ recordAuditEntry }));
|
||||
const user = await createUser();
|
||||
const res = buildRes();
|
||||
|
||||
await handlers.registerPasskeyVerify(
|
||||
authedReq(user, { credential: attestation, name: 'Work laptop', password: PASSWORD }),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(res.status).toHaveBeenCalledWith(201);
|
||||
const [stored] = await methods.findPasskeysByUser(user._id.toString());
|
||||
expect(recordAuditEntry).toHaveBeenCalledTimes(1);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'passkey.registered',
|
||||
actor: expect.objectContaining({ type: 'user', id: user._id.toString() }),
|
||||
target: { type: 'passkey', id: stored.id, name: 'Work laptop' },
|
||||
metadata: { deviceType: 'singleDevice', backedUp: false },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('does not record passkey.registered when registration is rejected', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const handlers = createPasskeyHandlers(buildDeps({ recordAuditEntry }));
|
||||
const user = await createUser();
|
||||
const res = buildRes();
|
||||
|
||||
await handlers.registerPasskeyVerify(
|
||||
authedReq(user, { credential: attestation, name: 'Key', password: 'wrong' }),
|
||||
res,
|
||||
);
|
||||
|
||||
expect(res.status).toHaveBeenCalledWith(403);
|
||||
expect(recordAuditEntry).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('names an unnamed credential after its transports', async () => {
|
||||
const handlers = createPasskeyHandlers(buildDeps());
|
||||
const user = await createUser();
|
||||
|
|
@ -710,6 +776,33 @@ describe('passkey rename', () => {
|
|||
expect(res.json).toHaveBeenCalledWith({ message: 'Name is required' });
|
||||
});
|
||||
|
||||
it('records passkey.removed only when a credential was deleted', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const handlers = createPasskeyHandlers(buildDeps({ recordAuditEntry }));
|
||||
const owner = await createUser();
|
||||
const other = await createUser();
|
||||
const passkey = await createStoredPasskey(owner);
|
||||
|
||||
await handlers.removePasskey(
|
||||
authedReq(other, { password: PASSWORD }, { passkeyId: passkey.id }),
|
||||
buildRes(),
|
||||
);
|
||||
expect(recordAuditEntry).not.toHaveBeenCalled();
|
||||
|
||||
await handlers.removePasskey(
|
||||
authedReq(owner, { password: PASSWORD }, { passkeyId: passkey.id }),
|
||||
buildRes(),
|
||||
);
|
||||
expect(recordAuditEntry).toHaveBeenCalledTimes(1);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'passkey.removed',
|
||||
actor: expect.objectContaining({ type: 'user', id: owner._id.toString() }),
|
||||
target: { type: 'passkey', id: passkey.id },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('answers 404 for a credential the caller does not own', async () => {
|
||||
const handlers = createPasskeyHandlers(buildDeps());
|
||||
const owner = await createUser();
|
||||
|
|
|
|||
|
|
@ -7,6 +7,8 @@ import type { AppConfigUserLike, GetAppConfigOptions } from '~/app/service';
|
|||
import type { PasskeyConfig, PasskeyChallengeStore } from '~/auth/passkey';
|
||||
import type { ComparePasswordDeps } from '~/auth/password';
|
||||
import type { UserDocumentId } from '~/auth/verification';
|
||||
import type { RecordAuditEntry } from '~/admin/audit';
|
||||
import type { ServerRequest } from '~/types/http';
|
||||
import {
|
||||
getPasskeyConfig,
|
||||
isPasskeyEnabled,
|
||||
|
|
@ -18,6 +20,7 @@ import {
|
|||
createPasskeyAuthenticationOptions,
|
||||
} from '~/auth/passkey';
|
||||
import { grandfatherLegacyEmailVerification } from '~/auth/verification';
|
||||
import { createAdminAuditEmitter } from '~/admin/audit';
|
||||
import { resolveStrictAppConfig } from '~/app/service';
|
||||
import { comparePassword } from '~/auth/password';
|
||||
import { isEnabled } from '~/utils';
|
||||
|
|
@ -121,6 +124,9 @@ export interface PasskeyHandlersDeps {
|
|||
/** Resolves effective configuration for the authenticated principal. */
|
||||
getAppConfig?: (options: GetAppConfigOptions) => Promise<AppConfig | undefined>;
|
||||
compare: ComparePasswordDeps['compare'];
|
||||
/** Optional audit emission for enrolment and removal; fail-open unless `auditFailClosed`. */
|
||||
recordAuditEntry?: RecordAuditEntry;
|
||||
auditFailClosed?: boolean;
|
||||
/** The login ban middleware; reports an internal failure through `next(err)`. */
|
||||
checkBan: (
|
||||
req: PasskeyRequest<AuthenticationBody>,
|
||||
|
|
@ -250,8 +256,16 @@ export function createPasskeyHandlers(deps: PasskeyHandlersDeps): PasskeyHandler
|
|||
getAppConfig,
|
||||
countPasskeysByUser,
|
||||
findPasskeyByCredentialId,
|
||||
recordAuditEntry,
|
||||
auditFailClosed,
|
||||
} = deps;
|
||||
|
||||
const emitAudit = createAdminAuditEmitter({
|
||||
recordAuditEntry,
|
||||
failClosed: auditFailClosed,
|
||||
logPrefix: '[passkey]',
|
||||
});
|
||||
|
||||
const getChallengeStore = (): PasskeyChallengeStore =>
|
||||
createPasskeyChallengeStore(getChallengeCache());
|
||||
|
||||
|
|
@ -446,18 +460,35 @@ export function createPasskeyHandlers(deps: PasskeyHandlersDeps): PasskeyHandler
|
|||
* the write and rolling this credential back keeps the stored set within
|
||||
* the cap without a transaction, at the cost of rejecting the whole race.
|
||||
*/
|
||||
const auditRegistered = (metadata: Record<string, unknown> = {}) =>
|
||||
emitAudit(req as unknown as ServerRequest, {
|
||||
action: 'passkey.registered',
|
||||
severity: 'warning',
|
||||
target: { type: 'passkey', id: passkey.id, name: passkey.name },
|
||||
metadata: { deviceType: verified.deviceType, backedUp: verified.backedUp, ...metadata },
|
||||
});
|
||||
/** A credential that survives a failed rollback is a usable factor, so it is audited. */
|
||||
const rollBack = async () => {
|
||||
try {
|
||||
await deletePasskey(passkey.id, req.user.id);
|
||||
} catch (rollbackError) {
|
||||
await auditRegistered({ rollback: 'failed' });
|
||||
throw rollbackError;
|
||||
}
|
||||
};
|
||||
let countAfterWrite: number;
|
||||
try {
|
||||
countAfterWrite = await countPasskeysByUser(req.user.id);
|
||||
} catch (error) {
|
||||
await deletePasskey(passkey.id, req.user.id);
|
||||
await rollBack();
|
||||
throw error;
|
||||
}
|
||||
if (countAfterWrite > maxPasskeys) {
|
||||
await deletePasskey(passkey.id, req.user.id);
|
||||
await rollBack();
|
||||
return res.status(409).json({ message: 'Passkey limit reached' });
|
||||
}
|
||||
|
||||
await auditRegistered();
|
||||
return res.status(201).json({ passkey: serializePasskey(passkey) });
|
||||
} catch (err) {
|
||||
if (isDuplicateKeyError(err as MongoWriteError)) {
|
||||
|
|
@ -529,6 +560,11 @@ export function createPasskeyHandlers(deps: PasskeyHandlersDeps): PasskeyHandler
|
|||
if (!result?.deletedCount) {
|
||||
return res.status(404).json({ message: 'Passkey not found' });
|
||||
}
|
||||
await emitAudit(req as unknown as ServerRequest, {
|
||||
action: 'passkey.removed',
|
||||
severity: 'warning',
|
||||
target: { type: 'passkey', id: req.params.passkeyId },
|
||||
});
|
||||
return res.status(200).json({ message: 'Passkey deleted' });
|
||||
} catch (err) {
|
||||
logger.error('[removePasskey]', err);
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ export * from './mcp/oauth';
|
|||
export * from './mcp/auth';
|
||||
export * from './mcp/zod';
|
||||
export * from './mcp/errors';
|
||||
export * from './mcp/audit';
|
||||
export * from './mcp/loading';
|
||||
export * from './mcp/openid';
|
||||
export * from './mcp/cache';
|
||||
|
|
|
|||
193
packages/api/src/mcp/audit.spec.ts
Normal file
193
packages/api/src/mcp/audit.spec.ts
Normal file
|
|
@ -0,0 +1,193 @@
|
|||
import type { ServerRequest } from '~/types/http';
|
||||
import { createMCPServerAuditor, settleMCPCommit, diffMCPServerFields } from './audit';
|
||||
import { MCPCommittedWriteError } from '~/mcp/errors';
|
||||
|
||||
const req = { user: { id: 'user-1', name: 'Alice' }, headers: {} } as unknown as ServerRequest;
|
||||
|
||||
describe('diffMCPServerFields', () => {
|
||||
it('returns sorted names of changed user-input fields only', () => {
|
||||
expect(
|
||||
diffMCPServerFields(
|
||||
{ url: 'https://a', title: 'T', headers: { A: '1' }, dbId: 'x' },
|
||||
{ url: 'https://b', title: 'T', headers: { A: '2' }, dbId: 'y' },
|
||||
),
|
||||
).toEqual(['headers', 'url']);
|
||||
});
|
||||
|
||||
it('ignores key order inside record fields', () => {
|
||||
expect(
|
||||
diffMCPServerFields({ headers: { A: '1', B: '2' } }, { headers: { B: '2', A: '1' } }),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it('treats a missing side as all-new', () => {
|
||||
expect(diffMCPServerFields(undefined, { type: 'sse', url: 'https://a' })).toEqual([
|
||||
'type',
|
||||
'url',
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('createMCPServerAuditor', () => {
|
||||
it('lists no fields for a delete', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const audit = createMCPServerAuditor({ recordAuditEntry });
|
||||
|
||||
await audit(req, {
|
||||
action: 'mcp_server.deleted',
|
||||
serverName: 'srv',
|
||||
previous: { url: 'https://a', title: 'Srv' },
|
||||
});
|
||||
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'mcp_server.deleted',
|
||||
severity: 'warning',
|
||||
target: { type: 'mcp_server', id: 'srv', name: 'Srv' },
|
||||
metadata: { source: 'DB' },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('does not record a preserved OAuth secret as a change', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const audit = createMCPServerAuditor({ recordAuditEntry });
|
||||
|
||||
await audit(req, {
|
||||
action: 'mcp_server.updated',
|
||||
serverName: 'srv',
|
||||
previous: { title: 'Old', oauth: { client_id: 'id', client_secret: 'stored' } },
|
||||
next: { title: 'New', oauth: { client_id: 'id' } },
|
||||
});
|
||||
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ metadata: { source: 'DB', fields: 'title' } }),
|
||||
);
|
||||
});
|
||||
|
||||
it('does not record omitted optional OAuth binding fields as a change', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const audit = createMCPServerAuditor({ recordAuditEntry });
|
||||
|
||||
await audit(req, {
|
||||
action: 'mcp_server.updated',
|
||||
serverName: 'srv',
|
||||
previous: {
|
||||
title: 'Old',
|
||||
oauth: {
|
||||
client_id: 'id',
|
||||
client_secret: 'stored',
|
||||
revocation_endpoint: 'https://idp.example/revoke',
|
||||
token_endpoint_auth_methods_supported: ['client_secret_basic'],
|
||||
},
|
||||
},
|
||||
next: { title: 'New', oauth: { client_id: 'id' } },
|
||||
});
|
||||
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ metadata: { source: 'DB', fields: 'title' } }),
|
||||
);
|
||||
});
|
||||
|
||||
it('diffs against the persisted config, not the submission', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const audit = createMCPServerAuditor({ recordAuditEntry });
|
||||
const headers = { Authorization: 'Bearer {{MCP_API_KEY_srv}}' };
|
||||
|
||||
await audit(req, {
|
||||
action: 'mcp_server.updated',
|
||||
serverName: 'srv',
|
||||
previous: { title: 'Old', headers },
|
||||
next: { title: 'New', headers },
|
||||
});
|
||||
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ metadata: { source: 'DB', fields: 'title' } }),
|
||||
);
|
||||
});
|
||||
|
||||
it('marks the entry when the cache refresh failed after the commit', async () => {
|
||||
const recordAuditEntry = jest.fn().mockResolvedValue(undefined);
|
||||
const audit = createMCPServerAuditor({ recordAuditEntry });
|
||||
|
||||
await audit(req, {
|
||||
action: 'mcp_server.deleted',
|
||||
serverName: 'srv',
|
||||
previous: { title: 'Old' },
|
||||
postWriteFailed: true,
|
||||
});
|
||||
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
action: 'mcp_server.deleted',
|
||||
metadata: { source: 'DB', postWrite: 'failed' },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
describe('MCPCommittedWriteError', () => {
|
||||
it('keeps the persisted config out of serialized and logged errors', () => {
|
||||
const error = new MCPCommittedWriteError(
|
||||
{ oauth: { client_secret: 'super-secret' } },
|
||||
new Error('redis down'),
|
||||
);
|
||||
|
||||
expect(error.committed).toEqual({ oauth: { client_secret: 'super-secret' } });
|
||||
expect(JSON.stringify(error)).not.toContain('super-secret');
|
||||
expect(Object.keys(error)).not.toContain('committed');
|
||||
expect(Object.keys(error)).not.toContain('cause');
|
||||
expect(error.cause).toBeInstanceOf(Error);
|
||||
});
|
||||
});
|
||||
|
||||
describe('settleMCPCommit', () => {
|
||||
it('returns the committed result when the write succeeds', async () => {
|
||||
await expect(settleMCPCommit(async () => ({ title: 'x' }))).resolves.toEqual({
|
||||
committed: { title: 'x' },
|
||||
});
|
||||
});
|
||||
|
||||
it('returns the committed result together with an invalidation failure', async () => {
|
||||
const failure = new MCPCommittedWriteError({ title: 'x' }, new Error('redis down'));
|
||||
|
||||
const settled = await settleMCPCommit(async () => {
|
||||
throw failure;
|
||||
});
|
||||
|
||||
expect(settled.committed).toEqual({ title: 'x' });
|
||||
expect(settled.postWriteError).toBe(failure);
|
||||
});
|
||||
|
||||
it('rethrows any other failure as an uncommitted write', async () => {
|
||||
await expect(
|
||||
settleMCPCommit(async () => {
|
||||
throw new Error('write failed');
|
||||
}),
|
||||
).rejects.toThrow('write failed');
|
||||
});
|
||||
});
|
||||
|
||||
it('runs the post-commit steps before the audit write and still audits when they throw', async () => {
|
||||
const order: string[] = [];
|
||||
const recordAuditEntry = jest.fn().mockImplementation(async () => void order.push('audit'));
|
||||
const audit = createMCPServerAuditor({ recordAuditEntry });
|
||||
|
||||
await expect(
|
||||
audit(req, { action: 'mcp_server.deleted', serverName: 'srv' }, async () => {
|
||||
order.push('fence');
|
||||
throw new Error('fence failed');
|
||||
}),
|
||||
).rejects.toThrow('fence failed');
|
||||
|
||||
expect(order).toEqual(['fence', 'audit']);
|
||||
expect(recordAuditEntry).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ metadata: { source: 'DB', postWrite: 'failed' } }),
|
||||
);
|
||||
});
|
||||
|
||||
it('is a no-op without a recorder', async () => {
|
||||
await expect(
|
||||
createMCPServerAuditor({})(req, { action: 'mcp_server.created', serverName: 'srv' }),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
165
packages/api/src/mcp/audit.ts
Normal file
165
packages/api/src/mcp/audit.ts
Normal file
|
|
@ -0,0 +1,165 @@
|
|||
import { MCP_USER_INPUT_FIELDS } from 'librechat-data-provider';
|
||||
import type { AdminAuditEmitter, RecordAuditEntry } from '~/admin/audit';
|
||||
import { createAdminAuditEmitter } from '~/admin/audit';
|
||||
import { MCPCommittedWriteError } from '~/mcp/errors';
|
||||
|
||||
type MCPServerAction = 'mcp_server.created' | 'mcp_server.updated' | 'mcp_server.deleted';
|
||||
type ServerConfigLike = Record<string, unknown> | null | undefined;
|
||||
|
||||
export interface MCPServerAuditEvent {
|
||||
action: MCPServerAction;
|
||||
serverName: string;
|
||||
/** Config before the change; omitted on create. */
|
||||
previous?: ServerConfigLike;
|
||||
/** Config after the change; omitted on delete. */
|
||||
next?: ServerConfigLike;
|
||||
/** The write committed but refreshing the read caches failed afterwards. */
|
||||
postWriteFailed?: boolean;
|
||||
/** The write found nothing to change (for example a delete that lost a race), so no entry is recorded. */
|
||||
noop?: boolean;
|
||||
}
|
||||
|
||||
/** JSON with object keys sorted, so entry order never reads as a change. */
|
||||
function stableStringify(value: unknown): string {
|
||||
return JSON.stringify(value, (_key, val) =>
|
||||
val && typeof val === 'object' && !Array.isArray(val)
|
||||
? Object.fromEntries(Object.entries(val).sort(([a], [b]) => a.localeCompare(b)))
|
||||
: val,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Lists the user-input fields that differ between two server configs. Only
|
||||
* field names are returned, so headers, env values and credentials never reach
|
||||
* the audit trail.
|
||||
*/
|
||||
export function diffMCPServerFields(previous: ServerConfigLike, next: ServerConfigLike): string[] {
|
||||
const changed: string[] = [];
|
||||
for (const field of MCP_USER_INPUT_FIELDS) {
|
||||
const before = previous?.[field];
|
||||
const after = next?.[field];
|
||||
if (before === undefined && after === undefined) {
|
||||
continue;
|
||||
}
|
||||
if (stableStringify(before) !== stableStringify(after)) {
|
||||
changed.push(field);
|
||||
}
|
||||
}
|
||||
return changed.sort();
|
||||
}
|
||||
|
||||
/** Optional OAuth fields the data layer also keeps when an update omits them alongside the secret. */
|
||||
const OMITTED_OAUTH_BINDING_FIELDS = [
|
||||
'token_endpoint_auth_methods_supported',
|
||||
'revocation_endpoint',
|
||||
'revocation_endpoint_auth_methods_supported',
|
||||
] as const;
|
||||
|
||||
/**
|
||||
* An update that omits `oauth.client_secret` keeps the stored one, so the secret
|
||||
* is carried onto the incoming config before comparing; otherwise a title-only
|
||||
* edit would be recorded as an oauth change.
|
||||
*/
|
||||
function withPreservedOAuthSecret(
|
||||
previous: ServerConfigLike,
|
||||
next: ServerConfigLike,
|
||||
): ServerConfigLike {
|
||||
const before = previous?.oauth as Record<string, unknown> | undefined;
|
||||
const after = next?.oauth as Record<string, unknown> | undefined;
|
||||
if (!next || !before?.client_secret || !after || after.client_secret) {
|
||||
return next;
|
||||
}
|
||||
const carried = { ...after, client_secret: before.client_secret } as Record<string, unknown>;
|
||||
for (const field of OMITTED_OAUTH_BINDING_FIELDS) {
|
||||
if (carried[field] === undefined && before[field] !== undefined) {
|
||||
carried[field] = before[field];
|
||||
}
|
||||
}
|
||||
return { ...next, oauth: carried };
|
||||
}
|
||||
|
||||
function serverTitle(config: ServerConfigLike): string | undefined {
|
||||
const title = config?.title;
|
||||
return typeof title === 'string' && title.length > 0 ? title : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds the MCP server audit writer. Fail-open unless `failClosed` is set,
|
||||
* matching the admin handler factories.
|
||||
*/
|
||||
export function createMCPServerAuditor(options: {
|
||||
recordAuditEntry?: RecordAuditEntry;
|
||||
failClosed?: boolean;
|
||||
}): (
|
||||
req: Parameters<AdminAuditEmitter>[0],
|
||||
event: MCPServerAuditEvent,
|
||||
afterCommit?: () => Promise<void>,
|
||||
) => Promise<void> {
|
||||
const emit = createAdminAuditEmitter({ ...options, logPrefix: '[mcpServers]' });
|
||||
async function record(
|
||||
req: Parameters<AdminAuditEmitter>[0],
|
||||
event: MCPServerAuditEvent,
|
||||
): Promise<void> {
|
||||
const { action, serverName, previous, next, postWriteFailed } = event;
|
||||
const name = serverTitle(next) ?? serverTitle(previous);
|
||||
const before = action === 'mcp_server.created' ? undefined : previous;
|
||||
const changed =
|
||||
action === 'mcp_server.deleted'
|
||||
? []
|
||||
: diffMCPServerFields(before, withPreservedOAuthSecret(before, next));
|
||||
await emit(req, {
|
||||
action,
|
||||
severity: action === 'mcp_server.deleted' ? 'warning' : 'info',
|
||||
target: { type: 'mcp_server', id: serverName, ...(name ? { name } : {}) },
|
||||
metadata: {
|
||||
source: 'DB',
|
||||
...(changed.length > 0 ? { fields: changed.join(',') } : {}),
|
||||
...(postWriteFailed ? { postWrite: 'failed' } : {}),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* `afterCommit` carries the security-critical steps that follow a committed
|
||||
* change (connection fences); they run first, and the audit write follows even
|
||||
* when they throw, so audit latency never delays them and a failure never
|
||||
* drops the entry.
|
||||
*/
|
||||
return async function auditMCPServer(req, event, afterCommit) {
|
||||
let afterCommitFailed = false;
|
||||
try {
|
||||
await afterCommit?.();
|
||||
} catch (error) {
|
||||
afterCommitFailed = true;
|
||||
throw error;
|
||||
} finally {
|
||||
if (!event.noop) {
|
||||
await record(req, {
|
||||
...event,
|
||||
postWriteFailed: event.postWriteFailed || afterCommitFailed,
|
||||
});
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs a registry write and reports what it persisted. When the write committed but the
|
||||
* cache refresh after it failed, the committed result is still returned together with that
|
||||
* failure, so the caller can audit the change and then surface the error.
|
||||
*/
|
||||
export async function settleMCPCommit<T>(
|
||||
write: () => Promise<T>,
|
||||
): Promise<{ committed: T | undefined; postWriteError?: MCPCommittedWriteError<T> }> {
|
||||
try {
|
||||
return { committed: await write() };
|
||||
} catch (error) {
|
||||
if (error instanceof MCPCommittedWriteError) {
|
||||
return {
|
||||
committed: error.committed as T | undefined,
|
||||
postWriteError: error as MCPCommittedWriteError<T>,
|
||||
};
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
|
@ -527,3 +527,24 @@ export function createScheduledMCPTransportError(
|
|||
agentId,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Thrown when a registry write committed but a step after it failed (permission
|
||||
* bookkeeping, cache invalidation). The write stands; `committed` carries what was
|
||||
* persisted (undefined for a removal) so a caller can still record it before surfacing
|
||||
* the failure. The persisted config can hold decrypted secrets, so the property is
|
||||
* non-enumerable and never reaches a serialized or logged error.
|
||||
*/
|
||||
export class MCPCommittedWriteError<T = unknown> extends Error {
|
||||
public readonly committed!: T | undefined;
|
||||
|
||||
constructor(committed: T | undefined, cause: unknown) {
|
||||
/** Fixed text: this message reaches API responses, and the cause may carry driver or cache details. */
|
||||
super('MCP server change was saved but a step after it failed');
|
||||
this.name = 'MCPCommittedWriteError';
|
||||
Object.defineProperty(this, 'committed', { value: committed, enumerable: false });
|
||||
/** Non-enumerable, so structured loggers never serialize driver or cache details. */
|
||||
Object.defineProperty(this, 'cause', { value: cause, enumerable: false, configurable: true });
|
||||
Object.setPrototypeOf(this, MCPCommittedWriteError.prototype);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -14,7 +14,11 @@ import {
|
|||
APP_CACHE_NAMESPACE,
|
||||
CONFIG_CACHE_NAMESPACE,
|
||||
} from './cache/ServerConfigsCacheFactory';
|
||||
import { MCPInspectionFailedError, isMCPDomainNotAllowedError } from '~/mcp/errors';
|
||||
import {
|
||||
MCPCommittedWriteError,
|
||||
MCPInspectionFailedError,
|
||||
isMCPDomainNotAllowedError,
|
||||
} from '~/mcp/errors';
|
||||
import { normalizeLegacyHeaderMaps, normalizeLegacyHeaderMapsIn } from './compat';
|
||||
import { canBackfillSharedServerInstructions, isUserSourced } from '~/mcp/utils';
|
||||
import { ReadThroughAllCache } from './cache/ReadThroughAllCache';
|
||||
|
|
@ -23,6 +27,7 @@ import { requireApiKeyReentryForRebinding } from './binding';
|
|||
import { ReadThroughCache } from './cache/ReadThroughCache';
|
||||
import { MCPServerInspector } from './MCPServerInspector';
|
||||
import { ServerConfigsDB } from './db/ServerConfigsDB';
|
||||
import { getSafeErrorMetadata } from '~/utils/errors';
|
||||
import { cacheConfig } from '~/cache';
|
||||
import { withTimeout } from '~/utils';
|
||||
|
||||
|
|
@ -672,22 +677,73 @@ export class MCPServersRegistry {
|
|||
...parsedConfig,
|
||||
source,
|
||||
};
|
||||
const result =
|
||||
storageLocation === 'DB'
|
||||
? await this.dbConfigsRepo.add(
|
||||
serverName,
|
||||
tagged,
|
||||
userId,
|
||||
await this.getOperatorManagedServerNames(reservedServerNames),
|
||||
)
|
||||
: await configRepo.add(serverName, tagged, userId);
|
||||
await this.invalidateServerReadCaches(result.serverName, userId, storageLocation);
|
||||
let result: t.AddServerResult;
|
||||
try {
|
||||
result =
|
||||
storageLocation === 'DB'
|
||||
? await this.dbConfigsRepo.add(
|
||||
serverName,
|
||||
tagged,
|
||||
userId,
|
||||
await this.getOperatorManagedServerNames(reservedServerNames),
|
||||
)
|
||||
: await configRepo.add(serverName, tagged, userId);
|
||||
} catch (error) {
|
||||
await this.invalidateAfterCommittedFailure(error, serverName, userId, storageLocation);
|
||||
throw error;
|
||||
}
|
||||
await this.invalidateCommitted(result, () =>
|
||||
this.invalidateServerReadCaches(result.serverName, userId, storageLocation),
|
||||
);
|
||||
if (storageLocation === 'CACHE') {
|
||||
this.resetYamlServerNamesMemo();
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* A repository step after a committed write failed (for example permission bookkeeping),
|
||||
* so the stored config changed even though the call rejects: refresh the read caches
|
||||
* best-effort before the failure propagates, or they would keep serving the old state.
|
||||
*/
|
||||
private async invalidateAfterCommittedFailure(
|
||||
error: unknown,
|
||||
serverName: string,
|
||||
userId: string | undefined,
|
||||
storageLocation: 'CACHE' | 'DB',
|
||||
): Promise<void> {
|
||||
if (!(error instanceof MCPCommittedWriteError)) {
|
||||
return;
|
||||
}
|
||||
const committed = error.committed as t.AddServerResult | undefined;
|
||||
await this.invalidateServerReadCaches(
|
||||
committed?.serverName ?? serverName,
|
||||
userId,
|
||||
storageLocation,
|
||||
).catch((invalidationError) =>
|
||||
logger.error(
|
||||
'[MCPServersRegistry] cache invalidation failed after a committed write error:',
|
||||
getSafeErrorMetadata(invalidationError),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs the cache invalidation that follows a committed write. A failure is rethrown as
|
||||
* `MCPCommittedWriteError` carrying what was persisted, so the caller can record
|
||||
* the committed change instead of mistaking it for a write that never happened.
|
||||
*/
|
||||
private async invalidateCommitted<T>(
|
||||
committed: T | undefined,
|
||||
invalidate: () => Promise<void>,
|
||||
): Promise<void> {
|
||||
try {
|
||||
await invalidate();
|
||||
} catch (error) {
|
||||
throw new MCPCommittedWriteError(committed, error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Backfills the inspector-derived `resolvedInstructions` for a server whose
|
||||
* operator explicitly deferred startup inspection. An enabled
|
||||
|
|
@ -1059,9 +1115,11 @@ export class MCPServersRegistry {
|
|||
userId?: string,
|
||||
): Promise<t.ParsedServerConfig> {
|
||||
const configRepo = this.getConfigRepository(storageLocation);
|
||||
await configRepo.update(serverName, parsedConfig, userId);
|
||||
await this.invalidateServerReadCaches(serverName, userId, storageLocation);
|
||||
return parsedConfig;
|
||||
const persisted = (await configRepo.update(serverName, parsedConfig, userId)) ?? parsedConfig;
|
||||
await this.invalidateCommitted(persisted, () =>
|
||||
this.invalidateServerReadCaches(serverName, userId, storageLocation),
|
||||
);
|
||||
return persisted;
|
||||
}
|
||||
|
||||
public async updateServer(
|
||||
|
|
@ -1323,13 +1381,22 @@ export class MCPServersRegistry {
|
|||
serverName: string,
|
||||
storageLocation: 'CACHE' | 'DB',
|
||||
userId?: string,
|
||||
): Promise<void> {
|
||||
): Promise<boolean> {
|
||||
const configRepo = this.getConfigRepository(storageLocation);
|
||||
await configRepo.remove(serverName, userId);
|
||||
await this.invalidateServerReadCaches(serverName, userId, storageLocation);
|
||||
let removed: boolean | void;
|
||||
try {
|
||||
removed = await configRepo.remove(serverName, userId);
|
||||
} catch (error) {
|
||||
await this.invalidateAfterCommittedFailure(error, serverName, userId, storageLocation);
|
||||
throw error;
|
||||
}
|
||||
await this.invalidateCommitted(removed !== false, () =>
|
||||
this.invalidateServerReadCaches(serverName, userId, storageLocation),
|
||||
);
|
||||
if (storageLocation === 'CACHE') {
|
||||
this.resetYamlServerNamesMemo();
|
||||
}
|
||||
return removed !== false;
|
||||
}
|
||||
|
||||
private getConfigRepository(storageLocation: 'CACHE' | 'DB'): IServerConfigsRepositoryInterface {
|
||||
|
|
|
|||
|
|
@ -7,7 +7,12 @@ export interface IServerConfigsRepositoryInterface {
|
|||
add(serverName: string, config: ParsedServerConfig, userId?: string): Promise<AddServerResult>;
|
||||
|
||||
//ACL Entry check if update is possible
|
||||
update(serverName: string, config: ParsedServerConfig, userId?: string): Promise<void>;
|
||||
/** Resolves to the config as persisted when the store can report it. */
|
||||
update(
|
||||
serverName: string,
|
||||
config: ParsedServerConfig,
|
||||
userId?: string,
|
||||
): Promise<ParsedServerConfig | void>;
|
||||
|
||||
/** Atomic add-or-update without requiring callers to inspect error messages. */
|
||||
upsert(serverName: string, config: ParsedServerConfig, userId?: string): Promise<void>;
|
||||
|
|
@ -45,7 +50,8 @@ export interface IServerConfigsRepositoryInterface {
|
|||
): Promise<ParsedServerConfig | undefined>;
|
||||
|
||||
//ACL Entry check if remove is possible
|
||||
remove(serverName: string, userId?: string): Promise<void>;
|
||||
/** Resolves to false when the store knew no such server, so a no-op is distinguishable. */
|
||||
remove(serverName: string, userId?: string): Promise<boolean | void>;
|
||||
|
||||
//ACL Entry check if read is possible
|
||||
get(serverName: string, userId?: string): Promise<ParsedServerConfig | undefined>;
|
||||
|
|
|
|||
|
|
@ -14,9 +14,9 @@ import {
|
|||
MCPConfigInitializationCanceledError,
|
||||
} from '~/mcp/registry/MCPServersRegistry';
|
||||
import { ServerConfigsCacheInMemory } from '~/mcp/registry/cache/ServerConfigsCacheInMemory';
|
||||
import { MCPCommittedWriteError, MCPInspectionFailedError } from '~/mcp/errors';
|
||||
import { getMCPAppToolsPublicationGeneration } from '~/mcp/toolsChanged';
|
||||
import { MCPServerInspector } from '~/mcp/registry/MCPServerInspector';
|
||||
import { MCPInspectionFailedError } from '~/mcp/errors';
|
||||
import { processMCPEnv } from '~/utils/env';
|
||||
|
||||
// Mock MCPServerInspector to avoid actual server connections
|
||||
|
|
@ -843,6 +843,67 @@ describe('MCPServersRegistry', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('cache invalidation failure after a committed write', () => {
|
||||
const failInvalidation = () =>
|
||||
jest
|
||||
.spyOn(
|
||||
registry as unknown as { invalidateServerReadCaches: () => Promise<void> },
|
||||
'invalidateServerReadCaches',
|
||||
)
|
||||
.mockRejectedValue(new Error('redis down'));
|
||||
|
||||
it('reports what addServer persisted', async () => {
|
||||
failInvalidation();
|
||||
|
||||
const failure = await registry
|
||||
.addServer('cache_server', testParsedConfig, 'CACHE')
|
||||
.catch((error) => error);
|
||||
|
||||
expect(failure).toBeInstanceOf(MCPCommittedWriteError);
|
||||
expect(failure.committed.serverName).toBe('cache_server');
|
||||
expect(await registry['cacheConfigsRepo'].get('cache_server')).toBeDefined();
|
||||
});
|
||||
|
||||
it('reports what commitServerUpdate persisted', async () => {
|
||||
await registry.addServer('cache_server', testParsedConfig, 'CACHE');
|
||||
failInvalidation();
|
||||
const updated = { ...testParsedConfig, command: 'python' } as t.ParsedServerConfig;
|
||||
|
||||
const failure = await registry
|
||||
.commitServerUpdate('cache_server', updated, 'CACHE')
|
||||
.catch((error) => error);
|
||||
|
||||
expect(failure).toBeInstanceOf(MCPCommittedWriteError);
|
||||
expect(failure.committed.command).toBe('python');
|
||||
});
|
||||
|
||||
it('refreshes the read caches when a repository step after the write fails', async () => {
|
||||
await registry.addServer('cache_server', testParsedConfig, 'CACHE');
|
||||
const invalidate = jest.spyOn(
|
||||
registry as unknown as { invalidateServerReadCaches: () => Promise<void> },
|
||||
'invalidateServerReadCaches',
|
||||
);
|
||||
jest
|
||||
.spyOn(registry['cacheConfigsRepo'], 'remove')
|
||||
.mockRejectedValueOnce(new MCPCommittedWriteError(undefined, new Error('acl down')));
|
||||
|
||||
await expect(registry.removeServer('cache_server', 'CACHE')).rejects.toBeInstanceOf(
|
||||
MCPCommittedWriteError,
|
||||
);
|
||||
expect(invalidate).toHaveBeenCalledWith('cache_server', undefined, 'CACHE');
|
||||
});
|
||||
|
||||
it('still reports a removal that committed', async () => {
|
||||
await registry.addServer('cache_server', testParsedConfig, 'CACHE');
|
||||
failInvalidation();
|
||||
|
||||
await expect(registry.removeServer('cache_server', 'CACHE')).rejects.toBeInstanceOf(
|
||||
MCPCommittedWriteError,
|
||||
);
|
||||
expect(await registry['cacheConfigsRepo'].get('cache_server')).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('Invalid storage location', () => {
|
||||
it('should throw error for unsupported storage location in addServer', async () => {
|
||||
await expect(
|
||||
|
|
|
|||
|
|
@ -204,6 +204,37 @@ describe('ServerConfigsDB', () => {
|
|||
expect(retrieved?.description).toBe('Updated description');
|
||||
});
|
||||
|
||||
it('returns the config as persisted, so an overlapping update is not mistaken for this one', async () => {
|
||||
const created = await serverConfigsDB.add(
|
||||
'temp-name',
|
||||
createSSEConfig('Title', 'First'),
|
||||
userId,
|
||||
);
|
||||
|
||||
const firstWrite = await serverConfigsDB.update(
|
||||
created.serverName,
|
||||
createSSEConfig('Title', 'Second'),
|
||||
userId,
|
||||
);
|
||||
await serverConfigsDB.update(created.serverName, createSSEConfig('Title', 'Third'), userId);
|
||||
|
||||
expect(firstWrite?.description).toBe('Second');
|
||||
expect((await serverConfigsDB.get(created.serverName, userId))?.description).toBe('Third');
|
||||
});
|
||||
|
||||
it('rejects an update when the server was deleted after the lookup', async () => {
|
||||
const created = await serverConfigsDB.add(
|
||||
'temp-name',
|
||||
createSSEConfig('Title', 'First'),
|
||||
userId,
|
||||
);
|
||||
await serverConfigsDB.remove(created.serverName, userId);
|
||||
|
||||
await expect(
|
||||
serverConfigsDB.update(created.serverName, createSSEConfig('Title', 'Second'), userId),
|
||||
).rejects.toThrow('no longer exists');
|
||||
});
|
||||
|
||||
it('should preserve oauth.client_secret when not provided in update', async () => {
|
||||
const config: ParsedServerConfig = {
|
||||
type: 'sse',
|
||||
|
|
@ -1427,12 +1458,40 @@ describe('ServerConfigsDB', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('post-write failures', () => {
|
||||
it('reports the created server when granting its owner access fails', async () => {
|
||||
jest
|
||||
.spyOn(serverConfigsDB['_aclService'], 'grantPermission')
|
||||
.mockRejectedValueOnce(new Error('acl down'));
|
||||
|
||||
const failure = await serverConfigsDB
|
||||
.add('temp-name', createSSEConfig('Orphan Risk'), userId)
|
||||
.catch((error) => error);
|
||||
|
||||
expect(failure.name).toBe('MCPCommittedWriteError');
|
||||
expect(failure.committed.config.title).toBe('Orphan Risk');
|
||||
expect(await mongoose.models.MCPServer.countDocuments({})).toBe(1);
|
||||
});
|
||||
|
||||
it('reports the removal when cleaning up its permissions fails', async () => {
|
||||
const created = await serverConfigsDB.add('temp-name', createSSEConfig('Remove Me'), userId);
|
||||
jest
|
||||
.spyOn(serverConfigsDB['_aclService'], 'removeAllPermissions')
|
||||
.mockRejectedValueOnce(new Error('acl down'));
|
||||
|
||||
await expect(serverConfigsDB.remove(created.serverName, userId)).rejects.toMatchObject({
|
||||
name: 'MCPCommittedWriteError',
|
||||
});
|
||||
expect(await mongoose.models.MCPServer.countDocuments({})).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('remove()', () => {
|
||||
it('should delete server from database', async () => {
|
||||
const config = createSSEConfig('Delete Test');
|
||||
const created = await serverConfigsDB.add('temp-name', config, userId);
|
||||
|
||||
await serverConfigsDB.remove(created.serverName, userId);
|
||||
await expect(serverConfigsDB.remove(created.serverName, userId)).resolves.toBe(true);
|
||||
|
||||
const MCPServer = mongoose.models.MCPServer;
|
||||
const server = await MCPServer.findOne({ serverName: created.serverName });
|
||||
|
|
@ -1461,8 +1520,8 @@ describe('ServerConfigsDB', () => {
|
|||
});
|
||||
|
||||
it('should handle non-existent server gracefully', async () => {
|
||||
// Should not throw
|
||||
await expect(serverConfigsDB.remove('non-existent-server', userId)).resolves.toBeUndefined();
|
||||
// Should not throw, and reports that nothing was removed
|
||||
await expect(serverConfigsDB.remove('non-existent-server', userId)).resolves.toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -11,8 +11,8 @@ import type { IServerConfigsRepositoryInterface } from '~/mcp/registry/ServerCon
|
|||
import type { ParsedServerConfig, AddServerResult } from '~/mcp/types';
|
||||
import type { ResolvedPrincipal } from '~/types/principal';
|
||||
import { getUserApiKeyVariable, requireApiKeyReentryForRebinding } from '~/mcp/registry/binding';
|
||||
import { MCPCommittedWriteError, MCPOAuthSecretReentryRequiredError } from '~/mcp/errors';
|
||||
import { normalizeLegacyHeaderMaps } from '~/mcp/registry/compat';
|
||||
import { MCPOAuthSecretReentryRequiredError } from '~/mcp/errors';
|
||||
import { AccessControlService } from '~/acl/accessControlService';
|
||||
import { isGeneratedUserApiKeyVariable } from '~/mcp/headers';
|
||||
|
||||
|
|
@ -316,18 +316,24 @@ export class ServerConfigsDB implements IServerConfigsRepositoryInterface {
|
|||
author: userId,
|
||||
reservedServerNames,
|
||||
});
|
||||
await this._aclService.grantPermission({
|
||||
principalType: PrincipalType.USER,
|
||||
principalId: userId,
|
||||
resourceType: ResourceType.MCPSERVER,
|
||||
resourceId: createdServer._id,
|
||||
accessRoleId: AccessRoleIds.MCPSERVER_OWNER,
|
||||
grantedBy: userId,
|
||||
});
|
||||
return {
|
||||
const created = {
|
||||
serverName: createdServer.serverName,
|
||||
config: await this.mapDBServerToParsedConfig(createdServer),
|
||||
};
|
||||
try {
|
||||
await this._aclService.grantPermission({
|
||||
principalType: PrincipalType.USER,
|
||||
principalId: userId,
|
||||
resourceType: ResourceType.MCPSERVER,
|
||||
resourceId: createdServer._id,
|
||||
accessRoleId: AccessRoleIds.MCPSERVER_OWNER,
|
||||
grantedBy: userId,
|
||||
});
|
||||
} catch (error) {
|
||||
/** The server exists already, so the failure must not read as a create that never happened. */
|
||||
throw new MCPCommittedWriteError(created, error);
|
||||
}
|
||||
return created;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -340,7 +346,7 @@ export class ServerConfigsDB implements IServerConfigsRepositoryInterface {
|
|||
serverName: string,
|
||||
config: ParsedServerConfig,
|
||||
userId?: string,
|
||||
): Promise<void> {
|
||||
): Promise<ParsedServerConfig> {
|
||||
if (!userId) {
|
||||
throw new Error(
|
||||
'[ServerConfigsDB.update] User ID is required to update a database-stored MCP server.',
|
||||
|
|
@ -405,7 +411,11 @@ export class ServerConfigsDB implements IServerConfigsRepositoryInterface {
|
|||
};
|
||||
}
|
||||
|
||||
await this._dbMethods.updateMCPServer(serverName, { config: configToSave });
|
||||
const saved = await this._dbMethods.updateMCPServer(serverName, { config: configToSave });
|
||||
if (!saved) {
|
||||
throw new Error(`[ServerConfigsDB.update] MCP server "${serverName}" no longer exists.`);
|
||||
}
|
||||
return await this.mapDBServerToParsedConfig(saved);
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -424,7 +434,7 @@ export class ServerConfigsDB implements IServerConfigsRepositoryInterface {
|
|||
'Config-source servers should use configCacheRepo, not dbConfigsRepo.',
|
||||
);
|
||||
}
|
||||
return this.update(serverName, config, userId);
|
||||
await this.update(serverName, config, userId);
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -432,18 +442,23 @@ export class ServerConfigsDB implements IServerConfigsRepositoryInterface {
|
|||
* @param serverName - The serverName of the server to remove
|
||||
* @param userId - User performing the deletion (for logging)
|
||||
*/
|
||||
public async remove(serverName: string, userId?: string): Promise<void> {
|
||||
public async remove(serverName: string, userId?: string): Promise<boolean> {
|
||||
logger.debug(`[ServerConfigsDB.remove] removing ${serverName}. UserId: ${userId}`);
|
||||
const deletedServer = await this._dbMethods.deleteMCPServer(serverName);
|
||||
if (deletedServer && deletedServer._id) {
|
||||
logger.debug(`[ServerConfigsDB.remove] removing all permissions entries of ${serverName}.`);
|
||||
await this._aclService.removeAllPermissions({
|
||||
resourceType: ResourceType.MCPSERVER,
|
||||
resourceId: deletedServer._id!,
|
||||
});
|
||||
return;
|
||||
try {
|
||||
await this._aclService.removeAllPermissions({
|
||||
resourceType: ResourceType.MCPSERVER,
|
||||
resourceId: deletedServer._id!,
|
||||
});
|
||||
} catch (error) {
|
||||
throw new MCPCommittedWriteError(true, error);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
logger.warn(`[ServerConfigsDB.remove] server with serverName ${serverName} does not exist`);
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -523,6 +523,59 @@ describe('auditLog methods', () => {
|
|||
expect(remaining.map((r) => r.seq)).toEqual([1, 2, 3, 4]);
|
||||
});
|
||||
|
||||
it('records audit.purged after a purge that removed rows, and not after a no-op', async () => {
|
||||
for (let i = 0; i < 4; i++) {
|
||||
await methods.recordAuditEntry(baseInput());
|
||||
await new Promise((resolve) => setTimeout(resolve, 5));
|
||||
}
|
||||
const third = await AuditLog.findOne({ chainKey: CK_A, seq: 3 }).lean();
|
||||
|
||||
await methods.purgeAuditLogEntries('tenant-a', {
|
||||
before: new Date(Date.now() - 60_000),
|
||||
confirm: true,
|
||||
});
|
||||
expect(await AuditLog.countDocuments({ chainKey: CK_A, action: 'audit.purged' })).toBe(0);
|
||||
|
||||
const result = await methods.purgeAuditLogEntries('tenant-a', {
|
||||
before: third!.createdAt,
|
||||
confirm: true,
|
||||
});
|
||||
|
||||
const entry = await AuditLog.findOne({ chainKey: CK_A, action: 'audit.purged' }).lean();
|
||||
expect(entry).toMatchObject({
|
||||
category: 'audit',
|
||||
severity: 'critical',
|
||||
actor: { type: 'system', name: 'audit-retention' },
|
||||
target: { type: 'audit_log' },
|
||||
metadata: { deletedCount: result.deletedCount, throughSeq: result.checkpoint?.throughSeq },
|
||||
});
|
||||
const verified = await methods.verifyAuditChain('tenant-a', {
|
||||
trustedCheckpoint: result.checkpoint,
|
||||
});
|
||||
expect(verified.ok).toBe(true);
|
||||
});
|
||||
|
||||
it('still returns the purge result when the audit write rejects', async () => {
|
||||
for (let i = 0; i < 4; i++) {
|
||||
await methods.recordAuditEntry(baseInput());
|
||||
await new Promise((resolve) => setTimeout(resolve, 5));
|
||||
}
|
||||
const third = await AuditLog.findOne({ chainKey: CK_A, seq: 3 }).lean();
|
||||
const spy = jest.spyOn(AuditLog, 'create').mockRejectedValue(new Error('audit store down'));
|
||||
|
||||
try {
|
||||
const result = await methods.purgeAuditLogEntries('tenant-a', {
|
||||
before: third!.createdAt,
|
||||
confirm: true,
|
||||
});
|
||||
expect(result.deletedCount).toBeGreaterThanOrEqual(1);
|
||||
expect(result.checkpoint).toBeDefined();
|
||||
} finally {
|
||||
spy.mockRestore();
|
||||
}
|
||||
expect(await AuditLog.countDocuments({ chainKey: CK_A, action: 'audit.purged' })).toBe(0);
|
||||
});
|
||||
|
||||
it('does not mint a checkpoint when a confirmed purge removed nothing', async () => {
|
||||
await seed(2);
|
||||
const result = await methods.purgeAuditLogEntries('tenant-a', {
|
||||
|
|
|
|||
|
|
@ -710,6 +710,29 @@ export function createAuditLogMethods(
|
|||
.select('seq prevHash')
|
||||
.lean<{ seq: number; prevHash: string }>();
|
||||
|
||||
/** The purge itself is audited after the checkpoint is read, so the new
|
||||
* entry cannot shift the boundary; a failed write is logged, not thrown. */
|
||||
try {
|
||||
await recordAuditEntry({
|
||||
action: 'audit.purged',
|
||||
severity: 'critical',
|
||||
actor: { type: 'system', name: 'audit-retention' },
|
||||
target: { type: 'audit_log' },
|
||||
metadata: {
|
||||
deletedCount,
|
||||
before: options.before.toISOString(),
|
||||
...(newFirst ? { throughSeq: newFirst.seq - 1 } : {}),
|
||||
},
|
||||
tenantId,
|
||||
});
|
||||
} catch (err) {
|
||||
logger.error('[auditLog] failed to record retention purge', {
|
||||
chainKey,
|
||||
deletedCount,
|
||||
error: err instanceof Error ? err.name : 'unknown error',
|
||||
});
|
||||
}
|
||||
|
||||
return {
|
||||
deletedCount,
|
||||
/** Boundary for `verifyAuditChain`: the new earliest entry resumes the
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue