mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-09-02 20:12:48 +00:00
* perf: cut serial round trips from the agent list query path
The agent list was the slowest path on first page load. Three separate
problems compounded:
- `getListAgentsHandler` chained its reads: two ACL lookups, the avatar
refresh cache probe and the viewer skill scope all resolved serially
ahead of the list query, and `attachOwnerContacts` added two more hops
after it. The four independent reads now resolve together, and the
avatar refresh runs alongside the list query instead of before it -
refreshed paths reach the response through `urlCache`, not through
whatever the list query happened to read. Serial hops per request drop
from 7 to 4 on a warm cache.
- The avatar refresh loaded the user's whole accessible agent set (up to
MAX_AVATAR_REFRESH_AGENTS) to discover which entries were S3-backed.
Scoping the query to `avatar.source` means deployments on any other
file strategy match nothing instead of walking the full set.
- `fetchAllAgentPages` walked cursor pages at the server's default size
of 100, and callers consume the flattened result, so every extra page
was a serial round trip for no benefit. It now requests the server
maximum. Measured over a 2,860 agent account: 29 requests / 1.65s
before, 3 requests / 0.29s after.
Also parallelizes the conversation file reads in `initializeAgent`. The
convo file refs and the execute_code thread walk share no inputs, and the
two code-file lookups depend only on `threadFileIds`, so the chain of six
serial reads on every turn collapses to two. This one is time to first
token the user waits through.
* perf: virtualize the model selector agent list
Opening the agents submenu with a large agent set froze the tab and could
kill it outright. With ~10k accessible agents the submenu blocked for over
15 seconds and took the heap from 96MB to 911MB. Four per-row costs were
being multiplied by the full list, which rendered unwindowed:
- `useIsActiveItem` allocated a MutationObserver per row (10,016 of them
for one dropdown). Replaced with an Ariakit store subscription, which
needs no observer at all and returns a boolean so a row only re-renders
when its own active state flips.
- `useFavorites` ran per row, opening a jotai subscription, a query
subscription and a mutation each time. Hoisted to one call per endpoint.
- Each row rescanned `endpoint.models` to recover `isGlobal`, a field the
parent had already discarded from the array it was mapping. The parent
now passes it down from a lookup map.
- The list itself is now windowed above 100 rows. Ariakit's composite only
knows about mounted rows, so arrow-keying to the window edge previously
found no next item and let focus escape the nested menu, closing it;
`handleBoundaryNavigation` scrolls the next index in, waits for it to
mount, then moves the composite onto it. Navigation inside the window is
left to Ariakit.
Open drops from >15s to 96ms, mounted rows from 10,028 to ~18, DOM nodes
from 123,346 to ~1,000, and the heap no longer grows. Verified in browser:
arrow keys track 1:1 to index 238 and back, and click selection works.
* perf: serve the model selector from the shared VIEW agent query
The model selector asked for EDIT-scoped agents whenever the marketplace
is enabled, while `useAgentsMap` and `useMentions` asked for VIEW. Since
the cache key includes the params, that was two distinct entries, so first
page load ran the paginated walk twice and held two copies of the whole
agent list in memory. Measured against a 10k agent account: 22 list handler
invocations per page load, now 11.
Collapsing the two by asking for the same permission everywhere would have
changed what the selector shows - under the marketplace the EDIT scope is
what makes it "My Agents", with discovery handled by the marketplace entry.
So the list endpoint now marks each row with `isEditable`, resolved from an
ACL read folded into the existing parallel batch (no extra serial hop), and
the selector filters the shared VIEW response instead of refetching. A
VIEW-scoped list for a user with 2861 visible / 361 editable agents returns
exactly 360 rows flagged editable, matching what the EDIT query returned.
`AgentSelect` deliberately keeps its own EDIT query: it reads `skills` and
`skills_enabled`, which `sanitizeViewerSkillScope` strips from VIEW-scoped
responses. It also only mounts when the builder panel is open, so it is not
part of the first-load cost.
The field is set unconditionally rather than omitted when false so that a
client talking to an older server sees `undefined`, keeps every agent, and
degrades to showing too many rather than none.
* fix: address review findings on the agent list at scale
Three issues from review, all confirmed against the code before fixing.
Avatar refresh no longer runs alongside the list query. `updateAgent` writes
through `findOneAndUpdate` on a `timestamps: true` schema, so refreshing an
avatar advances `updatedAt` — the field `getListAgentsByAccess` sorts and
cursors on. A write landing after the first page's snapshot moved that agent
ahead of the returned cursor, dropping it from every later page and silently
truncating the caller's flattened list. This was a regression introduced when
the two were parallelized; serializing them costs nothing on the common path,
because a cache hit returns without issuing any query, so only the
once-per-30-minutes miss pays for the ordering. The new test asserts the write
lands before the list snapshot and fails against the parallel version.
The virtualized list no longer inserts a focusable grid into the combobox.
`List` spreads its props onto `Grid`, whose defaults are `role="grid"`,
`containerRole="row"` and `tabIndex={0}`; inside Ariakit's listbox that added a
tab stop ahead of any row and put grid/row semantics between the listbox and its
options. All three are now neutralized so focus and ARIA stay with the combobox
items.
The list also resets to the top when the filter changes. `Grid` keeps its scroll
offset across prop changes and clamps an out-of-range offset to
`totalRowsHeight - height`, the end of the shorter list. Scrolling deep and then
searching landed on the tail: measured at row 626 of 667 matches, with only
those rows mounted and reachable by keyboard. Keying the list on the search
value restores row 0.
* fix: declare option position and set size for the virtualized model list
Once the model list is windowed, only the mounted slice exists in the listbox,
so a screen reader infers position and total from ~19 elements instead of the
real set — announcing "3 of 19" partway through 10,014 agents.
Model rows now carry aria-posinset and aria-setsize. The marketplace entry and
any model specs share the same numbering, because they are options in the same
listbox: declaring the values on some options while leaving others to be
inferred from the DOM would make the set internally inconsistent. Both are
omitted entirely when the list is short enough to render unwindowed, where the
DOM holds every option and the implicit values are already correct.
Verified against a 10,014 agent account: the marketplace entry reports 1 of
10015, the first models 2 and 3, and after scrolling to row 4999 the leading
mounted model reports 5001 of 10015 with 19 options in the DOM.
* 🩹 fix: Address Follow-Ups on the Agent List at Scale
Corrects residual issues in the agent-list perf work, all inside its own scope.
- Forward `idOnTheSource` through `PermissionService.findAccessibleResources`
so `getUserPrincipals` skips the user-document read. The list handler resolves
three permission sets per request and each was paying its own `User.findById`;
the auth strategies already normalize the field to a value or null.
- Gate the editable-set lookup on its own predicate instead of borrowing
`canReturnSkillConfig`. The two answer unrelated questions and only coincide
today, so redefining the skill flag would have marked every agent editable.
- Log mapping failures in the list response instead of swallowing them.
- Apply the walk page size after the caller's params in `fetchAllAgentPages`.
A caller limit only changed page size, never what the flattened walk returned,
so `defaultAgentParams`' `limit: 10` would have turned one request into 301.
- Carry `isEditable` on the agent rows the create and update mutations write
into the list cache. Mutation responses omit the field, so those rows lost it.
- Document `isEditable` as list-only, ACL-derived, and fail-open on absence.
- Restore the truthiness guard on the thread walk in `initializeAgent`. Widening
it to `!= null` made an empty `parentMessageId` issue a full-conversation read
against an anchor that can never match.
- Await `getConvoFiles` directly rather than calling `.then()` on it, restoring
tolerance for synchronous test doubles.
- Correct the avatar-refresh comment: the projection was never full documents,
and the real reason to filter is that an unfiltered budget is self-reinforcing.
Tests: both new `initialize` tests and both new backend tests are
mutation-verified; the concurrency test fails under either serialization order.
* fix: preserve ACL isEditable when merging agent mutation responses
Mutation responses omit list-only isEditable. Inferring true from write
success promoted VIEW-only rows into the editable subset for MANAGE_AGENTS
callers who can PATCH agents their ACL marks non-editable.
* fix: sort imports in agent mutations test
ESLint import-order check failed on the isEditable cache-preservation test.
* 🧷 fix: Carry isEditable Onto Duplicated Agent List Rows
`useDuplicateAgentMutation` prepended the raw duplicate response to the cached
list, and mutation responses omit the list-only `isEditable` field. The row
survived the "My Agents" filter only by failing open on `undefined`, so it would
disappear the moment a consumer read the flag strictly.
Duplicating grants the caller ownership, so the new row is editable outright;
this is the create case rather than the merge case `mergeAgentListRow` handles.
Last cache write on this path that did not carry the field.
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
994 lines
35 KiB
JavaScript
994 lines
35 KiB
JavaScript
const mongoose = require('mongoose');
|
|
const { isEnabled } = require('@librechat/api');
|
|
const {
|
|
getTransactionSupport,
|
|
tenantStorage,
|
|
getTenantId,
|
|
logger,
|
|
} = require('@librechat/data-schemas');
|
|
const { ResourceType, PrincipalType, PrincipalModel } = require('librechat-data-provider');
|
|
const {
|
|
entraIdPrincipalFeatureEnabled,
|
|
getUserOwnedEntraGroups,
|
|
getUserEntraGroups,
|
|
getEntraGroupDetailsBatch,
|
|
getGroupMembers,
|
|
getGroupOwners,
|
|
} = require('~/server/services/GraphApiService');
|
|
const db = require('~/models');
|
|
|
|
/** @type {boolean|null} */
|
|
let transactionSupportCache = null;
|
|
|
|
/**
|
|
* Validates that the resourceType is one of the supported enum values
|
|
* @param {string} resourceType - The resource type to validate
|
|
* @throws {Error} If resourceType is not valid
|
|
*/
|
|
const validateResourceType = (resourceType) => {
|
|
const validTypes = Object.values(ResourceType);
|
|
if (!validTypes.includes(resourceType)) {
|
|
throw new Error(`Invalid resourceType: ${resourceType}. Valid types: ${validTypes.join(', ')}`);
|
|
}
|
|
};
|
|
|
|
const ensureLocalUserPrincipalExists = async (principalId) => {
|
|
const user = await db.findUser({ _id: principalId }, '_id');
|
|
if (!user) {
|
|
throw new Error('User principal not found');
|
|
}
|
|
return user._id.toString();
|
|
};
|
|
|
|
const ensureLocalGroupPrincipalExists = async (principalId) => {
|
|
const group = await db.findGroupById(principalId, { _id: 1 });
|
|
if (!group) {
|
|
throw new Error('Group principal not found');
|
|
}
|
|
return group._id.toString();
|
|
};
|
|
|
|
/**
|
|
* @import { TPrincipal } from 'librechat-data-provider'
|
|
*/
|
|
/**
|
|
* Grant a permission to a principal for a resource using a role
|
|
* @param {Object} params - Parameters for granting role-based permission
|
|
* @param {string} params.principalType - PrincipalType.USER, PrincipalType.GROUP, or PrincipalType.PUBLIC
|
|
* @param {string|mongoose.Types.ObjectId|null} params.principalId - The ID of the principal (null for PrincipalType.PUBLIC)
|
|
* @param {string} params.resourceType - Type of resource (e.g., 'agent')
|
|
* @param {string|mongoose.Types.ObjectId} params.resourceId - The ID of the resource
|
|
* @param {string} params.accessRoleId - The ID of the role (e.g., AccessRoleIds.AGENT_VIEWER, AccessRoleIds.AGENT_EDITOR)
|
|
* @param {string|mongoose.Types.ObjectId} params.grantedBy - User ID granting the permission
|
|
* @param {mongoose.ClientSession} [params.session] - Optional MongoDB session for transactions
|
|
* @returns {Promise<Object>} The created or updated ACL entry
|
|
*/
|
|
const grantPermission = async ({
|
|
principalType,
|
|
principalId,
|
|
resourceType,
|
|
resourceId,
|
|
accessRoleId,
|
|
grantedBy,
|
|
session,
|
|
}) => {
|
|
try {
|
|
if (!Object.values(PrincipalType).includes(principalType)) {
|
|
throw new Error(`Invalid principal type: ${principalType}`);
|
|
}
|
|
|
|
if (principalType !== PrincipalType.PUBLIC && !principalId) {
|
|
throw new Error('Principal ID is required for user, group, and role principals');
|
|
}
|
|
|
|
// Validate principalId based on type
|
|
if (principalId && principalType === PrincipalType.ROLE) {
|
|
// Role IDs are strings (role names)
|
|
if (typeof principalId !== 'string' || principalId.trim().length === 0) {
|
|
throw new Error(`Invalid role ID: ${principalId}`);
|
|
}
|
|
} else if (
|
|
principalType &&
|
|
principalType !== PrincipalType.PUBLIC &&
|
|
!mongoose.Types.ObjectId.isValid(principalId)
|
|
) {
|
|
// User and Group IDs must be valid ObjectIds
|
|
throw new Error(`Invalid principal ID: ${principalId}`);
|
|
}
|
|
|
|
if (!resourceId || !mongoose.Types.ObjectId.isValid(resourceId)) {
|
|
throw new Error(`Invalid resource ID: ${resourceId}`);
|
|
}
|
|
|
|
validateResourceType(resourceType);
|
|
|
|
// Get the role to determine permission bits
|
|
const role = await db.findRoleByIdentifier(accessRoleId);
|
|
if (!role) {
|
|
throw new Error(`Role ${accessRoleId} not found`);
|
|
}
|
|
|
|
// Ensure the role is for the correct resource type
|
|
if (role.resourceType !== resourceType) {
|
|
throw new Error(
|
|
`Role ${accessRoleId} is for ${role.resourceType} resources, not ${resourceType}`,
|
|
);
|
|
}
|
|
return await db.grantPermission(
|
|
principalType,
|
|
principalId,
|
|
resourceType,
|
|
resourceId,
|
|
role.permBits,
|
|
grantedBy,
|
|
session,
|
|
role._id,
|
|
);
|
|
} catch (error) {
|
|
logger.error(`[PermissionService.grantPermission] Error: ${error.message}`);
|
|
throw error;
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Check if a user has specific permission bits on a resource
|
|
* @param {Object} params - Parameters for checking permissions
|
|
* @param {string|mongoose.Types.ObjectId} params.userId - The ID of the user
|
|
* @param {string} [params.role] - Optional user role (if not provided, will query from DB)
|
|
* @param {string} params.resourceType - Type of resource (e.g., 'agent')
|
|
* @param {string|mongoose.Types.ObjectId} params.resourceId - The ID of the resource
|
|
* @param {number} params.requiredPermissions - The permission bits required (e.g., 1 for VIEW, 3 for VIEW+EDIT)
|
|
* @returns {Promise<boolean>} Whether the user has the required permission bits
|
|
*/
|
|
const checkPermission = async ({ userId, role, resourceType, resourceId, requiredPermission }) => {
|
|
try {
|
|
if (typeof requiredPermission !== 'number' || requiredPermission < 1) {
|
|
throw new Error('requiredPermission must be a positive number');
|
|
}
|
|
|
|
validateResourceType(resourceType);
|
|
|
|
const principals = await db.getUserPrincipals({ userId, role });
|
|
|
|
if (principals.length === 0) {
|
|
return false;
|
|
}
|
|
|
|
return await db.hasPermission(principals, resourceType, resourceId, requiredPermission);
|
|
} catch (error) {
|
|
logger.error(`[PermissionService.checkPermission] Error: ${error.message}`);
|
|
if (error.message.includes('requiredPermission must be')) {
|
|
throw error;
|
|
}
|
|
return false;
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Get effective permission bitmask for a user on a resource
|
|
* @param {Object} params - Parameters for getting effective permissions
|
|
* @param {string|mongoose.Types.ObjectId} params.userId - The ID of the user
|
|
* @param {string} [params.role] - Optional user role (if not provided, will query from DB)
|
|
* @param {string} params.resourceType - Type of resource (e.g., 'agent')
|
|
* @param {string|mongoose.Types.ObjectId} params.resourceId - The ID of the resource
|
|
* @returns {Promise<number>} Effective permission bitmask
|
|
*/
|
|
const getEffectivePermissions = async ({ userId, role, resourceType, resourceId }) => {
|
|
try {
|
|
validateResourceType(resourceType);
|
|
|
|
const principals = await db.getUserPrincipals({ userId, role });
|
|
|
|
if (principals.length === 0) {
|
|
return 0;
|
|
}
|
|
|
|
return await db.getEffectivePermissions(principals, resourceType, resourceId);
|
|
} catch (error) {
|
|
logger.error(`[PermissionService.getEffectivePermissions] Error: ${error.message}`);
|
|
return 0;
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Get effective permissions for multiple resources in a batch operation
|
|
* Returns map of resourceId → effectivePermissionBits
|
|
*
|
|
* @param {Object} params - Parameters
|
|
* @param {string|mongoose.Types.ObjectId} params.userId - User ID
|
|
* @param {string} [params.role] - User role (for group membership)
|
|
* @param {string} params.resourceType - Resource type (must be valid ResourceType)
|
|
* @param {Array<mongoose.Types.ObjectId>} params.resourceIds - Array of resource IDs
|
|
* @returns {Promise<Map<string, number>>} Map of resourceId string → permission bits
|
|
* @throws {Error} If resourceType is invalid
|
|
*/
|
|
const getResourcePermissionsMap = async ({ userId, role, resourceType, resourceIds }) => {
|
|
// Validate resource type - throw on invalid type
|
|
validateResourceType(resourceType);
|
|
|
|
// Handle empty input
|
|
if (!Array.isArray(resourceIds) || resourceIds.length === 0) {
|
|
return new Map();
|
|
}
|
|
|
|
try {
|
|
// Get user principals (user + groups + public)
|
|
const principals = await db.getUserPrincipals({ userId, role });
|
|
|
|
// Use batch method from aclEntry
|
|
const permissionsMap = await db.getEffectivePermissionsForResources(
|
|
principals,
|
|
resourceType,
|
|
resourceIds,
|
|
);
|
|
|
|
logger.debug(
|
|
`[PermissionService.getResourcePermissionsMap] Computed permissions for ${resourceIds.length} resources, ${permissionsMap.size} have permissions`,
|
|
);
|
|
|
|
return permissionsMap;
|
|
} catch (error) {
|
|
logger.error(`[PermissionService.getResourcePermissionsMap] Error: ${error.message}`, error);
|
|
throw error;
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Find all resources of a specific type that a user has access to with specific permission bits
|
|
* @param {Object} params - Parameters for finding accessible resources
|
|
* @param {string|mongoose.Types.ObjectId} params.userId - The ID of the user
|
|
* @param {string} [params.role] - Optional user role (if not provided, will query from DB)
|
|
* @param {string|null} [params.idOnTheSource] - Optional external member id. `null` means "known to
|
|
* be absent" (local user); only `undefined` makes `getUserPrincipals` read the user document.
|
|
* @param {string} params.resourceType - Type of resource (e.g., 'agent')
|
|
* @param {number} params.requiredPermissions - The minimum permission bits required (e.g., 1 for VIEW, 3 for VIEW+EDIT)
|
|
* @returns {Promise<Array>} Array of resource IDs
|
|
*/
|
|
const findAccessibleResources = async ({
|
|
userId,
|
|
role,
|
|
idOnTheSource,
|
|
resourceType,
|
|
requiredPermissions,
|
|
}) => {
|
|
try {
|
|
if (typeof requiredPermissions !== 'number' || requiredPermissions < 1) {
|
|
throw new Error('requiredPermissions must be a positive number');
|
|
}
|
|
|
|
validateResourceType(resourceType);
|
|
|
|
// Get all principals for the user (user + groups + public)
|
|
const principalsList = await db.getUserPrincipals({ userId, role, idOnTheSource });
|
|
|
|
if (principalsList.length === 0) {
|
|
return [];
|
|
}
|
|
return await db.findAccessibleResources(principalsList, resourceType, requiredPermissions);
|
|
} catch (error) {
|
|
logger.error(`[PermissionService.findAccessibleResources] Error: ${error.message}`);
|
|
// Re-throw validation errors
|
|
if (error.message.includes('requiredPermissions must be')) {
|
|
throw error;
|
|
}
|
|
return [];
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Find all publicly accessible resources of a specific type
|
|
* @param {Object} params - Parameters for finding publicly accessible resources
|
|
* @param {string} params.resourceType - Type of resource (e.g., 'agent')
|
|
* @param {number} params.requiredPermissions - The minimum permission bits required (e.g., 1 for VIEW, 3 for VIEW+EDIT)
|
|
* @returns {Promise<Array>} Array of resource IDs
|
|
*/
|
|
const findPubliclyAccessibleResources = async ({ resourceType, requiredPermissions }) => {
|
|
try {
|
|
if (typeof requiredPermissions !== 'number' || requiredPermissions < 1) {
|
|
throw new Error('requiredPermissions must be a positive number');
|
|
}
|
|
|
|
validateResourceType(resourceType);
|
|
|
|
return await db.findPublicResourceIds(resourceType, requiredPermissions);
|
|
} catch (error) {
|
|
logger.error(`[PermissionService.findPubliclyAccessibleResources] Error: ${error.message}`);
|
|
if (error.message.includes('requiredPermissions must be')) {
|
|
throw error;
|
|
}
|
|
return [];
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Get available roles for a resource type
|
|
* @param {Object} params - Parameters for getting available roles
|
|
* @param {string} params.resourceType - Type of resource (e.g., 'agent')
|
|
* @returns {Promise<Array>} Array of role definitions
|
|
*/
|
|
const getAvailableRoles = async ({ resourceType }) => {
|
|
validateResourceType(resourceType);
|
|
|
|
return await db.findRolesByResourceType(resourceType);
|
|
};
|
|
|
|
/**
|
|
* Ensures a principal exists in the database based on TPrincipal data
|
|
* Creates user if it doesn't exist locally (for Entra ID users)
|
|
* @param {Object} principal - TPrincipal object from frontend
|
|
* @param {string} principal.type - PrincipalType.USER, PrincipalType.GROUP, or PrincipalType.PUBLIC
|
|
* @param {string} [principal.id] - Local database ID (null for Entra ID principals not yet synced)
|
|
* @param {string} principal.name - Display name
|
|
* @param {string} [principal.email] - Email address
|
|
* @param {string} [principal.source] - 'local' or 'entra'
|
|
* @param {string} [principal.idOnTheSource] - Entra ID object ID for external principals
|
|
* @returns {Promise<string|null>} Returns the principalId for database operations, null for public
|
|
*/
|
|
const ensurePrincipalExists = async function (principal) {
|
|
if (principal.type === PrincipalType.PUBLIC) {
|
|
return null;
|
|
}
|
|
|
|
if (principal.type === PrincipalType.USER && principal.id) {
|
|
return await ensureLocalUserPrincipalExists(principal.id);
|
|
}
|
|
|
|
if (principal.type === PrincipalType.USER && principal.source === 'entra') {
|
|
if (!principal.email || !principal.idOnTheSource) {
|
|
throw new Error('Entra ID user principals must have email and idOnTheSource');
|
|
}
|
|
|
|
let existingUser = await db.findUser({ idOnTheSource: principal.idOnTheSource });
|
|
|
|
if (!existingUser) {
|
|
existingUser = await db.findUser({ email: principal.email });
|
|
}
|
|
|
|
if (existingUser) {
|
|
if (!existingUser.idOnTheSource && principal.idOnTheSource) {
|
|
await db.updateUser(existingUser._id, {
|
|
idOnTheSource: principal.idOnTheSource,
|
|
provider: 'openid',
|
|
});
|
|
}
|
|
return existingUser._id.toString();
|
|
}
|
|
|
|
const userData = {
|
|
name: principal.name,
|
|
email: principal.email.toLowerCase(),
|
|
emailVerified: false,
|
|
provider: 'openid',
|
|
idOnTheSource: principal.idOnTheSource,
|
|
};
|
|
|
|
const userId = await db.createUser(userData, true, true);
|
|
return userId.toString();
|
|
}
|
|
|
|
if (principal.type === PrincipalType.GROUP) {
|
|
throw new Error('Group principals should be handled by group-specific methods');
|
|
}
|
|
|
|
throw new Error(`Unsupported principal type: ${principal.type}`);
|
|
};
|
|
|
|
/**
|
|
* Ensures a group principal exists in the database based on TPrincipal data
|
|
* Creates group if it doesn't exist locally (for Entra ID groups)
|
|
* For Entra ID groups, always synchronizes member IDs when authentication context is provided
|
|
* @param {Object} principal - TPrincipal object from frontend
|
|
* @param {string} principal.type - Must be PrincipalType.GROUP
|
|
* @param {string} [principal.id] - Local database ID (null for Entra ID principals not yet synced)
|
|
* @param {string} principal.name - Display name
|
|
* @param {string} [principal.email] - Email address
|
|
* @param {string} [principal.description] - Group description
|
|
* @param {string} [principal.source] - 'local' or 'entra'
|
|
* @param {string} [principal.idOnTheSource] - Entra ID object ID for external principals
|
|
* @param {Object} [authContext] - Optional authentication context for fetching member data
|
|
* @param {string} [authContext.accessToken] - Access token for Graph API calls
|
|
* @param {string} [authContext.sub] - Subject identifier
|
|
* @returns {Promise<string>} Returns the groupId for database operations
|
|
*/
|
|
const ensureGroupPrincipalExists = async function (principal, authContext = null) {
|
|
if (principal.type !== PrincipalType.GROUP) {
|
|
throw new Error(`Invalid principal type: ${principal.type}. Expected '${PrincipalType.GROUP}'`);
|
|
}
|
|
|
|
if (principal.id && principal.source !== 'entra') {
|
|
return await ensureLocalGroupPrincipalExists(principal.id);
|
|
}
|
|
|
|
if (principal.source === 'entra') {
|
|
if (!principal.name || !principal.idOnTheSource) {
|
|
throw new Error('Entra ID group principals must have name and idOnTheSource');
|
|
}
|
|
|
|
let memberIds = [];
|
|
if (authContext && authContext.accessToken && authContext.sub) {
|
|
try {
|
|
memberIds = await getGroupMembers(
|
|
authContext.accessToken,
|
|
authContext.sub,
|
|
principal.idOnTheSource,
|
|
);
|
|
|
|
// Include group owners as members if feature is enabled
|
|
if (isEnabled(process.env.ENTRA_ID_INCLUDE_OWNERS_AS_MEMBERS)) {
|
|
const ownerIds = await getGroupOwners(
|
|
authContext.accessToken,
|
|
authContext.sub,
|
|
principal.idOnTheSource,
|
|
);
|
|
if (ownerIds && ownerIds.length > 0) {
|
|
memberIds.push(...ownerIds);
|
|
// Remove duplicates
|
|
memberIds = [...new Set(memberIds)];
|
|
}
|
|
}
|
|
} catch (error) {
|
|
logger.error('Failed to fetch group members from Graph API:', error);
|
|
}
|
|
}
|
|
|
|
let existingGroup = await db.findGroupByExternalId(principal.idOnTheSource, 'entra');
|
|
|
|
if (!existingGroup && principal.email) {
|
|
existingGroup = await db.findGroupByQuery({ email: principal.email.toLowerCase() });
|
|
}
|
|
|
|
if (existingGroup) {
|
|
const updateData = {};
|
|
let needsUpdate = false;
|
|
|
|
if (!existingGroup.idOnTheSource && principal.idOnTheSource) {
|
|
updateData.idOnTheSource = principal.idOnTheSource;
|
|
updateData.source = 'entra';
|
|
needsUpdate = true;
|
|
}
|
|
|
|
if (principal.description && existingGroup.description !== principal.description) {
|
|
updateData.description = principal.description;
|
|
needsUpdate = true;
|
|
}
|
|
|
|
if (principal.email && existingGroup.email !== principal.email.toLowerCase()) {
|
|
updateData.email = principal.email.toLowerCase();
|
|
needsUpdate = true;
|
|
}
|
|
|
|
if (authContext && authContext.accessToken && authContext.sub) {
|
|
updateData.memberIds = memberIds;
|
|
needsUpdate = true;
|
|
}
|
|
|
|
if (needsUpdate) {
|
|
await db.updateGroupById(existingGroup._id, updateData);
|
|
}
|
|
|
|
return existingGroup._id.toString();
|
|
}
|
|
|
|
const groupData = {
|
|
name: principal.name,
|
|
source: 'entra',
|
|
idOnTheSource: principal.idOnTheSource,
|
|
memberIds: memberIds, // Store idOnTheSource values of group members (empty if no auth context)
|
|
};
|
|
|
|
if (principal.email) {
|
|
groupData.email = principal.email.toLowerCase();
|
|
}
|
|
|
|
if (principal.description) {
|
|
groupData.description = principal.description;
|
|
}
|
|
|
|
const newGroup = await db.createGroup(groupData);
|
|
return newGroup._id.toString();
|
|
}
|
|
if (principal.id && authContext == null) {
|
|
return principal.id;
|
|
}
|
|
|
|
throw new Error(`Unsupported group principal source: ${principal.source}`);
|
|
};
|
|
|
|
/**
|
|
* Sync user's Entra ID group memberships with auto-creation of missing groups
|
|
* Optimized approach:
|
|
* 1. Get all group IDs user should be member of from Entra
|
|
* 2. Try to add user to existing groups (fast, no Graph API calls)
|
|
* 3. Query DB to identify which groups don't exist (indexed query, fast)
|
|
* 4. For missing groups only, fetch details from Graph API in batches
|
|
* 5. Upsert missing groups using upsertGroupByExternalId (race-safe)
|
|
* 6. Add user to newly created/upserted groups via bulkUpdate
|
|
* 7. Remove user from groups they're no longer member of
|
|
*
|
|
* @param {Object} user - User object with authentication context
|
|
* @param {string} user.openidId - User's OpenID subject identifier
|
|
* @param {string} user.idOnTheSource - User's Entra ID (oid from token claims)
|
|
* @param {string} user.provider - Authentication provider ('openid')
|
|
* @param {string} accessToken - Access token for Graph API calls
|
|
* @param {mongoose.ClientSession} [session] - Optional MongoDB session for transactions
|
|
* @returns {Promise<void>}
|
|
*/
|
|
const syncUserEntraGroupMemberships = async (user, accessToken, session = null) => {
|
|
const tenantId = user?.tenantId ? String(user.tenantId) : undefined;
|
|
if (!tenantId || getTenantId() != null) {
|
|
return performEntraGroupMembershipSync(user, accessToken, session);
|
|
}
|
|
/**
|
|
* The OAuth callback runs before `tenantContextMiddleware`, so establish the
|
|
* user's tenant context here: group queries, created groups, and principal
|
|
* cache invalidation are then scoped exactly like authenticated reads.
|
|
*/
|
|
return tenantStorage.run({ tenantId, userId: user._id?.toString() }, async () =>
|
|
performEntraGroupMembershipSync(user, accessToken, session),
|
|
);
|
|
};
|
|
|
|
const performEntraGroupMembershipSync = async (user, accessToken, session = null) => {
|
|
try {
|
|
if (!entraIdPrincipalFeatureEnabled(user) || !accessToken || !user.idOnTheSource) {
|
|
return;
|
|
}
|
|
|
|
// Step 1: Get all group IDs user should be member of
|
|
const memberGroupIds = await getUserEntraGroups(accessToken, user.openidId);
|
|
let allGroupIds = [...(memberGroupIds || [])];
|
|
|
|
// Include owned groups if feature is enabled
|
|
if (isEnabled(process.env.ENTRA_ID_INCLUDE_OWNERS_AS_MEMBERS)) {
|
|
const ownedGroupIds = await getUserOwnedEntraGroups(accessToken, user.openidId);
|
|
if (ownedGroupIds && ownedGroupIds.length > 0) {
|
|
allGroupIds.push(...ownedGroupIds);
|
|
// Remove duplicates
|
|
allGroupIds = [...new Set(allGroupIds)];
|
|
}
|
|
}
|
|
|
|
const sessionOptions = session ? { session } : {};
|
|
|
|
// Early return if no groups found (protects against temporary API failures)
|
|
if (allGroupIds.length === 0) {
|
|
logger.debug(
|
|
`[PermissionService.syncUserEntraGroupMemberships] No groups found for user ${user._id}`,
|
|
);
|
|
return;
|
|
}
|
|
|
|
logger.info(
|
|
`[PermissionService.syncUserEntraGroupMemberships] Syncing ${allGroupIds.length} groups for user ${user._id}`,
|
|
);
|
|
|
|
// Step 2: Try to add user to existing groups (fast operation)
|
|
const addResult = await db.bulkUpdateGroups(
|
|
{
|
|
idOnTheSource: { $in: allGroupIds },
|
|
source: 'entra',
|
|
memberIds: { $ne: user.idOnTheSource },
|
|
},
|
|
{ $addToSet: { memberIds: user.idOnTheSource } },
|
|
sessionOptions,
|
|
);
|
|
|
|
logger.debug(
|
|
`[PermissionService.syncUserEntraGroupMemberships] Added user to ${addResult.modifiedCount || 0} existing groups`,
|
|
);
|
|
|
|
// Step 3: Find which groups don't exist in DB using db layer
|
|
const existingGroups = await db.findGroupsByExternalIds(allGroupIds, 'entra', session);
|
|
const existingGroupIds = new Set(existingGroups.map((g) => g.idOnTheSource));
|
|
|
|
const missingGroupIds = allGroupIds.filter((id) => !existingGroupIds.has(id));
|
|
|
|
if (missingGroupIds.length > 0) {
|
|
logger.info(
|
|
`[PermissionService.syncUserEntraGroupMemberships] Found ${missingGroupIds.length} groups that don't exist, fetching details...`,
|
|
);
|
|
|
|
// Step 4: Fetch details only for missing groups (optimized batch request)
|
|
const groupDetails = await getEntraGroupDetailsBatch(
|
|
accessToken,
|
|
user.openidId,
|
|
missingGroupIds,
|
|
);
|
|
|
|
if (groupDetails.length > 0) {
|
|
logger.info(
|
|
`[PermissionService.syncUserEntraGroupMemberships] Creating ${groupDetails.length} new groups`,
|
|
);
|
|
|
|
// Step 5: Upsert missing groups (race-safe by design)
|
|
// Use upsertGroupByExternalId for each group to handle concurrent creates gracefully
|
|
const upsertPromises = groupDetails.map((group) =>
|
|
db.upsertGroupByExternalId(
|
|
group.id,
|
|
'entra',
|
|
{
|
|
name: group.name,
|
|
email: group.email,
|
|
description: group.description,
|
|
},
|
|
session,
|
|
),
|
|
);
|
|
|
|
await Promise.all(upsertPromises);
|
|
|
|
// Step 6: Add user to all newly created/upserted groups
|
|
await db.bulkUpdateGroups(
|
|
{
|
|
idOnTheSource: { $in: missingGroupIds },
|
|
source: 'entra',
|
|
memberIds: { $ne: user.idOnTheSource },
|
|
},
|
|
{ $addToSet: { memberIds: user.idOnTheSource } },
|
|
sessionOptions,
|
|
);
|
|
|
|
logger.info(
|
|
`[PermissionService.syncUserEntraGroupMemberships] Successfully created/updated ${groupDetails.length} groups`,
|
|
);
|
|
} else {
|
|
logger.warn(
|
|
`[PermissionService.syncUserEntraGroupMemberships] Could not fetch details for ${missingGroupIds.length} missing groups`,
|
|
);
|
|
}
|
|
} else {
|
|
logger.debug(
|
|
`[PermissionService.syncUserEntraGroupMemberships] All ${allGroupIds.length} groups already exist in database`,
|
|
);
|
|
}
|
|
|
|
// Step 7: Remove user from Entra groups they're no longer member of
|
|
const removeResult = await db.bulkUpdateGroups(
|
|
{
|
|
source: 'entra',
|
|
memberIds: user.idOnTheSource,
|
|
idOnTheSource: { $nin: allGroupIds },
|
|
},
|
|
{ $pullAll: { memberIds: [user.idOnTheSource] } },
|
|
sessionOptions,
|
|
);
|
|
|
|
logger.debug(
|
|
`[PermissionService.syncUserEntraGroupMemberships] Removed user from ${removeResult.modifiedCount || 0} groups`,
|
|
);
|
|
|
|
logger.info(
|
|
`[PermissionService.syncUserEntraGroupMemberships] Successfully synced groups for user ${user._id}`,
|
|
);
|
|
} catch (error) {
|
|
// Log error but don't re-throw: group sync is best-effort operation
|
|
// and should not block authentication even if temporary API/DB issues occur
|
|
logger.error(`[PermissionService.syncUserEntraGroupMemberships] Error syncing groups:`, error);
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Check if public has a specific permission on a resource
|
|
* @param {Object} params - Parameters for checking public permission
|
|
* @param {string} params.resourceType - Type of resource (e.g., 'agent')
|
|
* @param {string|mongoose.Types.ObjectId} params.resourceId - The ID of the resource
|
|
* @param {number} params.requiredPermissions - The permission bits required (e.g., 1 for VIEW, 3 for VIEW+EDIT)
|
|
* @returns {Promise<boolean>} Whether public has the required permission bits
|
|
*/
|
|
const hasPublicPermission = async ({ resourceType, resourceId, requiredPermissions }) => {
|
|
try {
|
|
if (typeof requiredPermissions !== 'number' || requiredPermissions < 1) {
|
|
throw new Error('requiredPermissions must be a positive number');
|
|
}
|
|
|
|
validateResourceType(resourceType);
|
|
|
|
// Use public principal to check permissions
|
|
const publicPrincipal = [{ principalType: PrincipalType.PUBLIC }];
|
|
|
|
const entries = await db.findEntriesByPrincipalsAndResource(
|
|
publicPrincipal,
|
|
resourceType,
|
|
resourceId,
|
|
);
|
|
|
|
// Check if any entry has the required permission bits
|
|
return entries.some((entry) => (entry.permBits & requiredPermissions) === requiredPermissions);
|
|
} catch (error) {
|
|
logger.error(`[PermissionService.hasPublicPermission] Error: ${error.message}`);
|
|
// Re-throw validation errors
|
|
if (error.message.includes('requiredPermissions must be')) {
|
|
throw error;
|
|
}
|
|
return false;
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Bulk update permissions for a resource (grant, update, revoke)
|
|
* Efficiently handles multiple permission changes in a single transaction
|
|
*
|
|
* @param {Object} params - Parameters for bulk permission update
|
|
* @param {string} params.resourceType - Type of resource (e.g., 'agent')
|
|
* @param {string|mongoose.Types.ObjectId} params.resourceId - The ID of the resource
|
|
* @param {Array<TPrincipal>} params.updatedPrincipals - Array of principals to grant/update permissions for
|
|
* @param {Array<TPrincipal>} params.revokedPrincipals - Array of principals to revoke permissions from
|
|
* @param {string|mongoose.Types.ObjectId} params.grantedBy - User ID making the changes
|
|
* @param {mongoose.ClientSession} [params.session] - Optional MongoDB session for transactions
|
|
* @returns {Promise<Object>} Results object with granted, updated, revoked arrays and error details
|
|
*/
|
|
const bulkUpdateResourcePermissions = async ({
|
|
resourceType,
|
|
resourceId,
|
|
updatedPrincipals = [],
|
|
revokedPrincipals = [],
|
|
grantedBy,
|
|
session,
|
|
}) => {
|
|
const supportsTransactions = await getTransactionSupport(mongoose, transactionSupportCache);
|
|
transactionSupportCache = supportsTransactions;
|
|
let localSession = session;
|
|
let shouldEndSession = false;
|
|
|
|
try {
|
|
if (!Array.isArray(updatedPrincipals)) {
|
|
throw new Error('updatedPrincipals must be an array');
|
|
}
|
|
|
|
if (!Array.isArray(revokedPrincipals)) {
|
|
throw new Error('revokedPrincipals must be an array');
|
|
}
|
|
|
|
if (!resourceId || !mongoose.Types.ObjectId.isValid(resourceId)) {
|
|
throw new Error(`Invalid resource ID: ${resourceId}`);
|
|
}
|
|
|
|
if (!localSession && supportsTransactions) {
|
|
localSession = await mongoose.startSession();
|
|
localSession.startTransaction();
|
|
shouldEndSession = true;
|
|
}
|
|
|
|
const sessionOptions = localSession ? { session: localSession } : {};
|
|
|
|
const roles = await db.findRolesByResourceType(resourceType);
|
|
const rolesMap = new Map();
|
|
roles.forEach((role) => {
|
|
rolesMap.set(role.accessRoleId, role);
|
|
});
|
|
|
|
const results = {
|
|
granted: [],
|
|
updated: [],
|
|
revoked: [],
|
|
errors: [],
|
|
};
|
|
|
|
const bulkWrites = [];
|
|
|
|
/**
|
|
* Tracks non-public principals granted in this same request so their revoke is skipped below.
|
|
* Grants are flushed before deletes, so a principal present in both `updatedPrincipals` and
|
|
* `revokedPrincipals` would be upserted and then deleted, stripping access the caller just set
|
|
* (e.g. a resource owner landing in both lists from a client `id`/`idOnTheSource` mismatch).
|
|
* Granting wins to make owner lockout impossible regardless of the client-side diff (#14316).
|
|
*
|
|
* PUBLIC is deliberately excluded: an explicit `public: false` disable adds the public principal
|
|
* to the revoke list, and disabling public access must always win over a stale/contradictory
|
|
* grant so a resource is never left public when the caller asked to make it private.
|
|
*/
|
|
const grantedPrincipalKeys = new Set();
|
|
const principalKey = (principal) => `${principal.type}:${principal.id}`;
|
|
|
|
for (const principal of updatedPrincipals) {
|
|
try {
|
|
if (!principal.accessRoleId) {
|
|
results.errors.push({
|
|
principal,
|
|
error: 'accessRoleId is required for updated principals',
|
|
});
|
|
continue;
|
|
}
|
|
|
|
const role = rolesMap.get(principal.accessRoleId);
|
|
if (!role) {
|
|
results.errors.push({
|
|
principal,
|
|
error: `Role ${principal.accessRoleId} not found`,
|
|
});
|
|
continue;
|
|
}
|
|
|
|
const query = {
|
|
principalType: principal.type,
|
|
resourceType,
|
|
resourceId,
|
|
};
|
|
|
|
if (principal.type !== PrincipalType.PUBLIC) {
|
|
query.principalId =
|
|
principal.type === PrincipalType.ROLE
|
|
? principal.id
|
|
: new mongoose.Types.ObjectId(principal.id);
|
|
}
|
|
|
|
const principalModelMap = {
|
|
[PrincipalType.USER]: PrincipalModel.USER,
|
|
[PrincipalType.GROUP]: PrincipalModel.GROUP,
|
|
[PrincipalType.ROLE]: PrincipalModel.ROLE,
|
|
};
|
|
|
|
const update = {
|
|
$set: {
|
|
permBits: role.permBits,
|
|
roleId: role._id,
|
|
grantedBy,
|
|
grantedAt: new Date(),
|
|
},
|
|
$setOnInsert: {
|
|
principalType: principal.type,
|
|
resourceType,
|
|
resourceId,
|
|
...(principal.type !== PrincipalType.PUBLIC && {
|
|
principalId:
|
|
principal.type === PrincipalType.ROLE
|
|
? principal.id
|
|
: new mongoose.Types.ObjectId(principal.id),
|
|
principalModel: principalModelMap[principal.type],
|
|
}),
|
|
},
|
|
};
|
|
|
|
bulkWrites.push({
|
|
updateOne: {
|
|
filter: query,
|
|
update: update,
|
|
upsert: true,
|
|
},
|
|
});
|
|
|
|
results.granted.push({
|
|
type: principal.type,
|
|
id: principal.id,
|
|
name: principal.name,
|
|
email: principal.email,
|
|
source: principal.source,
|
|
avatar: principal.avatar,
|
|
description: principal.description,
|
|
idOnTheSource: principal.idOnTheSource,
|
|
accessRoleId: principal.accessRoleId,
|
|
memberCount: principal.memberCount,
|
|
memberIds: principal.memberIds,
|
|
});
|
|
if (principal.type !== PrincipalType.PUBLIC) {
|
|
grantedPrincipalKeys.add(principalKey(principal));
|
|
}
|
|
} catch (error) {
|
|
results.errors.push({
|
|
principal,
|
|
error: error.message,
|
|
});
|
|
}
|
|
}
|
|
|
|
if (bulkWrites.length > 0) {
|
|
await db.bulkWriteAclEntries(bulkWrites, sessionOptions);
|
|
}
|
|
|
|
const deleteQueries = [];
|
|
for (const principal of revokedPrincipals) {
|
|
try {
|
|
// Inside the try so a malformed revoke entry (e.g. a nullish principal) is recorded in
|
|
// results.errors and skipped, rather than throwing out after grants were already flushed.
|
|
if (
|
|
principal.type !== PrincipalType.PUBLIC &&
|
|
grantedPrincipalKeys.has(principalKey(principal))
|
|
) {
|
|
continue;
|
|
}
|
|
const query = {
|
|
principalType: principal.type,
|
|
resourceType,
|
|
resourceId,
|
|
};
|
|
|
|
if (principal.type !== PrincipalType.PUBLIC) {
|
|
query.principalId =
|
|
principal.type === PrincipalType.ROLE
|
|
? principal.id
|
|
: new mongoose.Types.ObjectId(principal.id);
|
|
}
|
|
|
|
deleteQueries.push(query);
|
|
|
|
results.revoked.push({
|
|
type: principal.type,
|
|
id: principal.id,
|
|
name: principal.name,
|
|
email: principal.email,
|
|
source: principal.source,
|
|
avatar: principal.avatar,
|
|
description: principal.description,
|
|
idOnTheSource: principal.idOnTheSource,
|
|
memberCount: principal.memberCount,
|
|
});
|
|
} catch (error) {
|
|
results.errors.push({
|
|
principal,
|
|
error: error.message,
|
|
});
|
|
}
|
|
}
|
|
|
|
if (deleteQueries.length > 0) {
|
|
await db.deleteAclEntries({ $or: deleteQueries }, sessionOptions);
|
|
}
|
|
|
|
if (shouldEndSession && supportsTransactions) {
|
|
await localSession.commitTransaction();
|
|
}
|
|
|
|
return results;
|
|
} catch (error) {
|
|
if (shouldEndSession && supportsTransactions) {
|
|
try {
|
|
await localSession.abortTransaction();
|
|
} catch (transactionError) {
|
|
/** best-effort abort; may fail if commit already succeeded */
|
|
logger.error(
|
|
`[PermissionService.bulkUpdateResourcePermissions] Error aborting transaction:`,
|
|
transactionError,
|
|
);
|
|
}
|
|
}
|
|
logger.error(`[PermissionService.bulkUpdateResourcePermissions] Error: ${error.message}`);
|
|
throw error;
|
|
} finally {
|
|
if (shouldEndSession && localSession) {
|
|
localSession.endSession();
|
|
}
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Remove all permissions for a resource (cleanup when resource is deleted)
|
|
* @param {Object} params - Parameters for removing all permissions
|
|
* @param {string} params.resourceType - Type of resource (e.g., 'agent', 'prompt')
|
|
* @param {string|mongoose.Types.ObjectId} params.resourceId - The ID of the resource
|
|
* @returns {Promise<Object>} Result of the deletion operation
|
|
*/
|
|
const removeAllPermissions = async ({ resourceType, resourceId }) => {
|
|
try {
|
|
validateResourceType(resourceType);
|
|
|
|
if (!resourceId || !mongoose.Types.ObjectId.isValid(resourceId)) {
|
|
throw new Error(`Invalid resource ID: ${resourceId}`);
|
|
}
|
|
|
|
const result = await db.deleteAclEntries({
|
|
resourceType,
|
|
resourceId,
|
|
});
|
|
|
|
return result;
|
|
} catch (error) {
|
|
logger.error(`[PermissionService.removeAllPermissions] Error: ${error.message}`);
|
|
throw error;
|
|
}
|
|
};
|
|
|
|
module.exports = {
|
|
grantPermission,
|
|
checkPermission,
|
|
getEffectivePermissions,
|
|
getResourcePermissionsMap,
|
|
findAccessibleResources,
|
|
findPubliclyAccessibleResources,
|
|
hasPublicPermission,
|
|
getAvailableRoles,
|
|
bulkUpdateResourcePermissions,
|
|
ensurePrincipalExists,
|
|
ensureGroupPrincipalExists,
|
|
syncUserEntraGroupMemberships,
|
|
removeAllPermissions,
|
|
};
|