🥪 feat: Compose Remote Model Admission With Output Protection (#18276)

* feat: compose remote model admission with output protection

* chore: adopt agents 4.2.9 for remote policy composition
This commit is contained in:
Danny Avila 2026-11-18 06:17:01 -05:00 • committed by GitHub
parent 4cd70c5af9
commit 7d1f618b24
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 328 additions and 156 deletions

View file

@ -153,27 +153,27 @@ acceptance target remains the main chat's functionality and experience, with mea
Using native events and the native renderer proves only the behaviors tested below. Missing
adapters and unverified behaviors remain release gates for products that depend on them.
| Main-chat capability | This bridge | Remaining work / verification |
| ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Saved agent identity, role authorization, picker | Reuses the existing registration and picker | Remote enrollment/discovery is not implemented; operators configure deployments. |
| Text and tool-step rendering, persistence | Verified with a minimal Run and a custom graph in real LibreChat | Deterministic models were used; repeat with production providers and representative workloads. |
| Reasoning, citations, rich/custom tool events | Native event transport is available | Provider-specific and rich-content end-to-end parity has not been verified. |
| Custom graph, nested SDK node, conditional refinement | Verified in another process | Broader graphs, parallel branches, and large checkpoints need workload acceptance tests. |
| Approval after browser reload and worker restart | Approve verified with host Mongo checkpoints | Edit/reject, ask-user questions, and repeated/batched interruptions need remote browser tests. |
| Stop and browser reconnection to an active job | Verified; stop reaches the worker abort signal and persists no error | Cancellation is cooperative and cannot undo completed tool effects. |
| Host-worker connection recovery | Opt-in same-worker live reattachment; durable terminal replay after restart | No host-process adoption, live cross-worker migration, or arbitrary in-flight graph restart. Started executions never automatically rerun. |
| Model/PII/provider-text protection | Host-model chat supports input/PII callbacks and initial persistence admission; explicit provider-text transport remains available | Combining initial admission with provider-text output protection requires SDK invocation-context support. Protected Responses/OpenAI callers without native callbacks remain gated. |
| Stream circuit breakers and tool-policy hooks | Forwarded to the worker; hook failure stops execution | Domain tools must also authorize signed caller identity; trusted factories must retain the supplied adapters. |
| Agent-builder model, parameters, tools, capabilities | Opt-in saved-agent primary model settings execute on the host; host schema tools are available | Worker must retain the host model/profile. Fallbacks, local graph tools, and context/summary policy still require adapters. |
| Host MCP, web/code/browser tools, user tool credentials | Authorized root-agent schema tools execute through host RPC | Host credentials remain local; child-agent authorization and broader host-service parity remain separate integrations. |
| Uploaded files, generated artifacts, download authorization | Message data can cross the wire | Host file services, sandbox storage, and artifact authorization do not have remote adapters; attachment/file parity is not established. |
| Live steering and mid-stream preemption | Not integrated | A remote execution-control adapter and acceptance tests are required. |
| Foreground/nested/detached subagents | Worker may construct its own SDK graph | Native configured compositions are rejected; detached child checkpoint threads/context/usage sinks are not integrated. |
| Context pruning, compaction, summaries, cache/fading policy | Worker can use SDK features; terminal context metadata is returned | Host configuration and continuation policy are not automatically installed on the worker. Main-chat behavioral/performance parity is not established. |
| Token usage, cost/balance accounting | Native model events can be transported | Arbitrary worker model attribution and nested usage/balance enforcement need an explicit contract and tests; do not assume billing parity. |
| Tracing and tenant trace/export/redaction policy | Worker can configure SDK tracing itself | Host tracing callbacks and tenant export policy are not automatically inherited. Correlated, policy-safe tracing remains an integration gap. |
| Schedules/background/Responses callers | The bridge returns a Run through the common execution seam | Full lifecycle acceptance for these callers has not been performed. |
| Console Assistant and Schema Advisor | No product cutover in this change | Wire Schema Advisor, close its required gaps, then adopt shared chat components and compare performance. |
| Main-chat capability | This bridge | Remaining work / verification |
| ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Saved agent identity, role authorization, picker | Reuses the existing registration and picker | Remote enrollment/discovery is not implemented; operators configure deployments. |
| Text and tool-step rendering, persistence | Verified with a minimal Run and a custom graph in real LibreChat | Deterministic models were used; repeat with production providers and representative workloads. |
| Reasoning, citations, rich/custom tool events | Native event transport is available | Provider-specific and rich-content end-to-end parity has not been verified. |
| Custom graph, nested SDK node, conditional refinement | Verified in another process | Broader graphs, parallel branches, and large checkpoints need workload acceptance tests. |
| Approval after browser reload and worker restart | Approve verified with host Mongo checkpoints | Edit/reject, ask-user questions, and repeated/batched interruptions need remote browser tests. |
| Stop and browser reconnection to an active job | Verified; stop reaches the worker abort signal and persists no error | Cancellation is cooperative and cannot undo completed tool effects. |
| Host-worker connection recovery | Opt-in same-worker live reattachment; durable terminal replay after restart | No host-process adoption, live cross-worker migration, or arbitrary in-flight graph restart. Started executions never automatically rerun. |
| Model/PII/provider-text protection | Host-model chat supports input/PII callbacks and initial persistence admission; explicit provider-text transport remains available | Combined admission/output protection uses SDK 4.2.9 invocation context and capability negotiation. Older workers remain gated; protected API callers without native callbacks still need integration. |
| Stream circuit breakers and tool-policy hooks | Forwarded to the worker; hook failure stops execution | Domain tools must also authorize signed caller identity; trusted factories must retain the supplied adapters. |
| Agent-builder model, parameters, tools, capabilities | Opt-in saved-agent primary model settings execute on the host; host schema tools are available | Worker must retain the host model/profile. Fallbacks, local graph tools, and context/summary policy still require adapters. |
| Host MCP, web/code/browser tools, user tool credentials | Authorized root-agent schema tools execute through host RPC | Host credentials remain local; child-agent authorization and broader host-service parity remain separate integrations. |
| Uploaded files, generated artifacts, download authorization | Message data can cross the wire | Host file services, sandbox storage, and artifact authorization do not have remote adapters; attachment/file parity is not established. |
| Live steering and mid-stream preemption | Not integrated | A remote execution-control adapter and acceptance tests are required. |
| Foreground/nested/detached subagents | Worker may construct its own SDK graph | Native configured compositions are rejected; detached child checkpoint threads/context/usage sinks are not integrated. |
| Context pruning, compaction, summaries, cache/fading policy | Worker can use SDK features; terminal context metadata is returned | Host configuration and continuation policy are not automatically installed on the worker. Main-chat behavioral/performance parity is not established. |
| Token usage, cost/balance accounting | Native model events can be transported | Arbitrary worker model attribution and nested usage/balance enforcement need an explicit contract and tests; do not assume billing parity. |
| Tracing and tenant trace/export/redaction policy | Worker can configure SDK tracing itself | Host tracing callbacks and tenant export policy are not automatically inherited. Correlated, policy-safe tracing remains an integration gap. |
| Schedules/background/Responses callers | The bridge returns a Run through the common execution seam | Full lifecycle acceptance for these callers has not been performed. |
| Console Assistant and Schema Advisor | No product cutover in this change | Wire Schema Advisor, close its required gaps, then adopt shared chat components and compare performance. |
Do not enable a workflow on the assumption that a native chat control is honored remotely. Its
required rows must be implemented and verified first. Keep this matrix current as adapters land;
@ -266,7 +266,7 @@ trusted code and must install it on every applicable SDK Run. This output-text a
supply input/model callbacks, authoritative builder model selection, host model
credentials, or policy for model calls made outside the SDK. SDK 4.2.7 also provides
`HostChatModel` and `createHostModelService` for host-owned model execution; those
contracts have the authenticated transport below. The input-admission adapter and this output-protection adapter currently cannot be combined, as explained below.
contracts have the authenticated transport below. Combining the input-admission and output-protection adapters requires SDK invocation context and the additional worker capability described below.
The worker buffers a complete model attempt through the same SDK release boundary
as local execution, then sends one `policy.providerText` RPC. Only canonical approved
@ -416,11 +416,29 @@ the registration uses the injected host model. Thus protected chat can opt in, w
Responses/OpenAI-compatible callers without that callback remain gated. Local graph tools and
provider fallbacks retain their earlier explicit rejection.
**Remaining limitation:** the SDK provider-text output-protection wrapper deliberately withholds
raw callback managers from its underlying model. Until the SDK preserves a separate invocation
identity, combining that wrapper with initial remote admission is rejected on both sides before
graph execution. Existing programmatic output protection without an initial barrier continues to
work. Do not disable output protection to enable this combination.
### Combining input admission and output protection
With [SDK 4.2.9 invocation-context support](https://github.com/LibreChat-AI/agents/pull/704),
the worker uses the native `HostChatModel` service context to correlate each model
attempt with its root graph callback. The context contains only a version and run ID;
raw callback managers remain suppressed by the output-protection boundary. Correlation
still requires an enrolled root callback and the authenticated host's observed native
success. Input rejection prevents provider execution; output rejection prevents ordinary
deferred persistence. Approved output reaches native events and persistence once.
This combination requires the additive `modelAdmissionWithProviderText: 1` worker
capability, advertised only when its installed SDK supports invocation context. The
host checks it before factory execution, and the worker independently rejects unsupported
combined requests. `modelAdmission: 1` is unchanged, preserving older host behavior.
LibreChat depends on SDK 4.2.9 or later. Upgrade separately deployed workers to SDK 4.2.9
and this bridge version to enable combined mode. SDK 4.2.8 retains the existing input-only
adapter and does **not** enable the combination; old workers fail closed rather than
weakening protection.
Callers still explicitly supply the host provider-text service; this negotiation does
not automatically install product output policies or enable protected API callers lacking
native input callbacks. Separate-process and live-recovery tests cover the combined service
path. Provider-specific UI acceptance, billing and tracing remain separate work.
Before initial admission, child/summary model calls without root lineage and concurrent root
attempts fail closed; parallel/composed admission needs its own ownership contract. This does not

View file

@ -171,6 +171,8 @@ consume `input.agent.model` and retain `input.hostModel`; model credentials stay
The custom graph keeps its plan/review/refine nodes and uses the host-authorized tool schemas.
This mode requires updated workers with `builderModel: 1` and, for initial persistence admission,
`modelAdmission: 1`. Protected chat uses native host input callbacks. Protected API callers without
those callbacks, local graph tools, provider fallbacks, and the combination of initial admission
with provider-text output protection remain gated; see [the parity matrix](../../docs/sdk/agents.md#main-chat-parity-and-rollout-gates)
those callbacks, local graph tools and provider fallbacks remain gated. Combining initial admission
with explicit provider-text output protection requires `modelAdmissionWithProviderText: 1`, advertised
only with SDK 4.2.9 invocation-context support. Upgrade separately deployed workers to SDK 4.2.9
and this bridge version; older workers stay gated. See [the parity matrix](../../docs/sdk/agents.md#main-chat-parity-and-rollout-gates)
and [configuration](../../docs/sdk/agents.md#saved-agent-builder-models) before enabling it.

View file

@ -821,7 +821,8 @@ endpoints:
# # Optional primary-model execution for remote SDK agents. Budgets are per process.
# # Restart to change budgets; workers must support builderModel: 1.
# # Initial protected-chat admission also requires modelAdmission: 1 on workers.
# # Local graph tools, provider fallbacks and combined output protection remain gated.
# # Combined output protection requires modelAdmissionWithProviderText: 1 and an explicit policy service.
# # Local graph tools and provider fallbacks remain gated.
# sdkHostModels:
# maxConcurrentCalls: 32
# maxAdmissionEntries: 4096

8
package-lock.json generated
View file

@ -9479,9 +9479,9 @@
}
},
"node_modules/@librechat/agents": {
"version": "4.2.8",
"resolved": "https://registry.npmjs.org/@librechat/agents/-/agents-4.2.8.tgz",
"integrity": "sha512-I26rtj6ODcNqN9hzqFIQN4Ma/v1z5ShI6ew8jmRqZMev1nzIjLLYQeWn4p8gUi+NXE+Zm3n4bdsUouQ7LU3MGQ==",
"version": "4.2.9",
"resolved": "https://registry.npmjs.org/@librechat/agents/-/agents-4.2.9.tgz",
"integrity": "sha512-0i896o9zFNg3I/yxMfrCFRCGspA+HGlt0e2Ng+2YA1MHC3rA/TQ5LqSGqheuBaixfXs8CbcMVKJF7MnZFVdQTQ==",
"license": "MIT",
"dependencies": {
"@anthropic-ai/sdk": "^0.115.0",
@ -42445,7 +42445,7 @@
"@azure/identity": "^4.14.0",
"@azure/storage-blob": "^12.30.0",
"@keyv/redis": "5.1.6",
"@librechat/agents": "^4.2.8",
"@librechat/agents": "^4.2.9",
"@librechat/data-schemas": "*",
"@modelcontextprotocol/sdk": "^1.32.1",
"@opentelemetry/api": "^1.9.0",

View file

@ -120,7 +120,7 @@
"@azure/identity": "^4.14.0",
"@azure/storage-blob": "^12.30.0",
"@keyv/redis": "5.1.6",
"@librechat/agents": "^4.2.8",
"@librechat/agents": "^4.2.9",
"@librechat/data-schemas": "*",
"@modelcontextprotocol/sdk": "^1.32.1",
"@opentelemetry/api": "^1.9.0",

View file

@ -4,7 +4,15 @@ import {
createInitialModelBoundAdmissionCallback,
createModelBoundChatModelCallback,
} from '../../middleware/modelBoundContent';
import { createHostModelLifecycle, createWorkerModelLifecycle } from './admission';
import {
createHostModelLifecycle,
createWorkerModelLifecycle,
supportsModelInvocationContext,
} from './admission';
it('installs an SDK that exercises combined admission and output protection', () => {
expect(supportsModelInvocationContext).toBe(true);
});
function fixture() {
const persist = jest.fn();

View file

@ -18,6 +18,12 @@ export const modelAttemptSchema: z.ZodType<RemoteModelAttempt> = z
.object({ nodeId: z.string().uuid(), modelId: z.string().uuid() })
.strict();
const nodeEndSchema = z.object({ nodeId: z.string().uuid(), completed: z.boolean() }).strict();
const invocationContextSchema = z
.object({ version: z.literal(1), runId: z.string().uuid() })
.strict();
/** Additive SDK capability: older SDKs keep the input-only model adapter during rollout. */
export const supportsModelInvocationContext: boolean =
'invocationContextVersion' in HostChatModel && HostChatModel.invocationContextVersion === 1;
const blocked = () => new HostModelServiceError('blocked');
/** Run each iterator operation inside its scope; constructing an async generator does not enter it. */
@ -248,7 +254,7 @@ export function createHostModelLifecycle(agentId: string, maxEntries: number): H
export interface WorkerModelLifecycle {
callback: BaseCallbackHandler;
current(): RemoteModelAttempt | undefined;
current(context?: unknown): RemoteModelAttempt | undefined;
model(options: ConstructorParameters<typeof HostChatModel>[0]): HostChatModel;
}
@ -310,13 +316,23 @@ export function createWorkerModelLifecycle(
}
}
return {
current() {
current(context) {
if (admitted) return undefined;
if (supportsModelInvocationContext) {
const { runId } = invocationContextSchema.parse(context);
const attempt = models.get(runId);
if (!attempt) throw blocked();
attemptedNodes.add(attempt.nodeId);
return attempt;
}
const value = storage.getStore();
if (!value) throw blocked();
return value;
},
model: (options) => new CorrelatedHostModel(options),
model: (options) =>
supportsModelInvocationContext
? new HostChatModel(options)
: new CorrelatedHostModel(options),
callback: ensureHandler({
name: 'librechat-remote-model-admission',
raiseError: true,

View file

@ -41,8 +41,8 @@ import {
createInitialModelBoundAdmissionCallback,
createModelBoundChatModelCallback,
} from '../../middleware/modelBoundContent';
import { createHostModelLifecycle, supportsModelInvocationContext } from './admission';
import { createSdkAgentRun, resolveSdkAgentBinding } from '../sdk';
import { createHostModelLifecycle } from './admission';
import { createRun as createHostRun } from '../run';
import { createRemoteSdkServer } from './server';
import { createSdkModelHost } from './builder';
@ -1223,7 +1223,13 @@ describe('remote host provider-text protection', () => {
});
});
it.each(['providerText', 'model', 'builderModel', 'modelAdmission'] as const)(
it.each([
'providerText',
'model',
'builderModel',
'modelAdmission',
'modelAdmissionWithProviderText',
] as const)(
'rejects a worker without %s capability before factory admission',
async (capability) => {
const server = new WebSocketServer({ port: 0, host: '127.0.0.1' });
@ -1241,6 +1247,9 @@ it.each(['providerText', 'model', 'builderModel', 'modelAdmission'] as const)(
return {
transportVersion: 2,
hostTools: 2,
...(capability === 'modelAdmissionWithProviderText'
? { providerText: 1, modelAdmission: 1, hostModel: 1, builderModel: 1 }
: {}),
...(['builderModel', 'modelAdmission'].includes(capability)
? { hostModel: 1, ...(capability === 'modelAdmission' ? { builderModel: 1 } : {}) }
: {}),
@ -1269,7 +1278,8 @@ it.each(['providerText', 'model', 'builderModel', 'modelAdmission'] as const)(
? { providerText }
: {
model: nativeModelBinding(),
...(capability === 'modelAdmission'
...(capability === 'modelAdmissionWithProviderText' ? { providerText } : {}),
...(['modelAdmission', 'modelAdmissionWithProviderText'].includes(capability)
? { modelLifecycle: createHostModelLifecycle(identity.agentId, 16) }
: {}),
...(capability === 'builderModel'
@ -1586,105 +1596,206 @@ function admissionCallback(initial: InitialModelBoundAdmissionCallback) {
}
describe('remote native policy admission', () => {
it('rejects combined output protection and initial admission before worker execution', async () => {
const createRun = jest.fn(factory);
const runtime = await fixture(createRun);
try {
it.each(['allowed', 'input-blocked', 'output-blocked'] as const)(
'negotiates combined policy and admits only approved output (%s)',
async (outcome) => {
const persisted = jest.fn();
const nativeCall = jest.fn();
const visible: string[] = [];
const inspect = jest.fn(() => {
expect(persisted).not.toHaveBeenCalled();
return outcome === 'output-blocked'
? { version: 1 as const, ok: false as const, error: { code: 'blocked' as const } }
: {
version: 1 as const,
ok: true as const,
value: { content: 'approved', replacements: 1, categories: [] },
};
});
const createRun = jest.fn(async (input: RemoteSdkFactoryInput) => {
const run = await factory(input);
run.Graph!.overrideModel = input.hostModel;
return run;
});
const runtime = await fixture(createRun);
const log = jest.spyOn(console, 'error').mockImplementation(() => {});
try {
const native = new TestModel({});
Object.assign(native, {
callbacks: [
createModelBoundChatModelCallback({
filters: {
messages: {
pii: {
fields: ['text'],
starterPatterns: [],
customPatterns: [{ id: 'private', label: 'Private', regex: 'PRIVATE-INPUT' }],
},
},
},
}),
],
});
native._streamResponseChunks = async function* () {
nativeCall();
yield new ChatGenerationChunk({
text: 'PRIVATE-OUTPUT',
message: new AIMessageChunk('PRIVATE-OUTPUT'),
});
};
const config = hostConfig();
config.customHandlers = {
[GraphEvents.ON_MESSAGE_DELTA]: {
handle: async (_event, data) => {
visible.push(JSON.stringify(data));
},
},
};
const host = createSdkModelHost(
agentsEndpointSchema.parse({ sdkHostModels: {} }).sdkHostModels,
() => native,
)!;
const initial = createInitialModelBoundAdmissionCallback({
agentIds: [identity.agentId],
isActive: () => true,
onAllowed: persisted,
});
const run = await createRemoteSdkRun({
config,
identity,
remote: runtime.remote,
hostServices: {
...host.bind(config, identity),
providerText: createHostProviderTextService({
identity,
policy: {
version: 1,
timeoutMs: 1000,
maxAttemptBytes: 4096,
maxBufferedBytes: 8192,
classify: () => 'prose',
inspect,
},
}),
},
});
const result = run.processStream(
{ messages: [new HumanMessage(outcome === 'input-blocked' ? 'PRIVATE-INPUT' : 'hello')] },
{ ...invocation(), callbacks: [admissionCallback(initial)] },
);
if (!supportsModelInvocationContext) {
await expect(result).rejects.toThrow('SDK runtime execution failed');
expect(createRun).not.toHaveBeenCalled();
expect(nativeCall).not.toHaveBeenCalled();
expect(inspect).not.toHaveBeenCalled();
expect(persisted).not.toHaveBeenCalled();
} else if (outcome === 'allowed') {
const content = await result;
expect(JSON.stringify(content)).toContain('approved');
expect(persisted).toHaveBeenCalledTimes(1);
expect(nativeCall).toHaveBeenCalledTimes(1);
expect(inspect).toHaveBeenCalledTimes(1);
expect(visible.join('')).toContain('approved');
expect(JSON.stringify(run.getRunMessages())).not.toContain('PRIVATE-OUTPUT');
} else {
await expect(result).rejects.toThrow();
expect(persisted).not.toHaveBeenCalled();
expect(nativeCall).toHaveBeenCalledTimes(outcome === 'input-blocked' ? 0 : 1);
expect(inspect).toHaveBeenCalledTimes(outcome === 'input-blocked' ? 0 : 1);
}
expect(visible.join('')).not.toContain('PRIVATE-OUTPUT');
} finally {
log.mockRestore();
await runtime.close();
}
},
);
it.each([false, true])(
'does not persist cancelled partial output (combined=%s)',
async (combined) => {
const persisted = jest.fn();
const initial = createInitialModelBoundAdmissionCallback({
agentIds: [identity.agentId],
isActive: () => true,
onAllowed: jest.fn(),
onAllowed: persisted,
});
const run = await createRemoteSdkRun({
config: hostConfig(),
identity,
remote: runtime.remote,
hostServices: {
model: nativeModelBinding(),
modelLifecycle: createHostModelLifecycle(identity.agentId, 16),
providerText: createHostProviderTextService({
identity,
policy: {
version: 1,
timeoutMs: 1000,
maxAttemptBytes: 4096,
maxBufferedBytes: 8192,
classify: () => 'prose',
inspect: () => ({ version: 1, ok: false, error: { code: 'blocked' } }),
},
}),
},
const abort = new AbortController();
let entered!: () => void;
let release!: () => void;
const started = new Promise<void>((resolve) => {
entered = resolve;
});
await expect(
run.processStream(
{ messages: [] },
{ ...invocation(), callbacks: [admissionCallback(initial)] },
),
).rejects.toThrow('SDK invocation context support');
expect(createRun).not.toHaveBeenCalled();
} finally {
await runtime.close();
}
});
it('does not persist a deferred message when a partially streamed model is cancelled', async () => {
const persisted = jest.fn();
const initial = createInitialModelBoundAdmissionCallback({
agentIds: [identity.agentId],
isActive: () => true,
onAllowed: persisted,
});
const abort = new AbortController();
let entered!: () => void;
let release!: () => void;
const started = new Promise<void>((resolve) => {
entered = resolve;
});
const pending = new Promise<void>((resolve) => {
release = resolve;
});
const runtime = await fixture(async (input) => {
const run = await factory(input);
run.Graph!.overrideModel = input.hostModel;
return run;
});
try {
const model = new TestModel({});
model._streamResponseChunks = async function* () {
yield new ChatGenerationChunk({
text: 'partial',
message: new AIMessageChunk({ content: 'partial' }),
});
entered();
await pending;
};
const run = await createSdkAgentRun({
config: hostConfig(),
identity,
signal: abort.signal,
registration: {
binding: { id: 'custom', protocolVersion: 1 },
remote: { ...runtime.remote, hostModel: true },
},
modelHost: createSdkModelHost(
const pending = new Promise<void>((resolve) => {
release = resolve;
});
const runtime = await fixture(async (input) => {
const run = await factory(input);
run.Graph!.overrideModel = input.hostModel;
return run;
});
try {
const model = new TestModel({});
model._streamResponseChunks = async function* () {
yield new ChatGenerationChunk({
text: 'partial',
message: new AIMessageChunk({ content: 'partial' }),
});
entered();
await pending;
};
const config = hostConfig();
const modelHost = createSdkModelHost(
agentsEndpointSchema.parse({ sdkHostModels: {} }).sdkHostModels,
() => model,
),
});
const execution = run.processStream(
{ messages: [new HumanMessage('safe')] },
{ ...invocation(), signal: abort.signal, callbacks: [admissionCallback(initial)] },
);
void execution.catch(() => {});
await started;
abort.abort();
release();
await expect(execution).rejects.toThrow('cancelled');
expect(persisted).not.toHaveBeenCalled();
} finally {
release();
await runtime.close();
}
});
)!;
const inspect = jest.fn(() => ({
version: 1 as const,
ok: true as const,
value: { content: 'approved', replacements: 0, categories: [] },
}));
const run = await createRemoteSdkRun({
config,
identity,
remote: runtime.remote,
hostServices: {
...modelHost.bind(config, identity, abort.signal),
...(combined && {
providerText: createHostProviderTextService({
identity,
policy: {
version: 1,
timeoutMs: 1000,
maxAttemptBytes: 4096,
maxBufferedBytes: 8192,
classify: () => 'prose',
inspect,
},
}),
}),
},
});
const execution = run.processStream(
{ messages: [new HumanMessage('safe')] },
{ ...invocation(), signal: abort.signal, callbacks: [admissionCallback(initial)] },
);
void execution.catch(() => {});
if (combined && !supportsModelInvocationContext) {
await expect(execution).rejects.toThrow('SDK runtime execution failed');
expect(persisted).not.toHaveBeenCalled();
return;
}
await started;
abort.abort();
release();
await expect(execution).rejects.toThrow('cancelled');
expect(inspect).not.toHaveBeenCalled();
expect(persisted).not.toHaveBeenCalled();
} finally {
release();
await runtime.close();
}
},
);
it.each([false, true])(
'admits deferred persistence only after the safe model node (blocked=%s)',
async (blocked) => {

View file

@ -201,12 +201,6 @@ export async function createRemoteSdkRun({
return socket;
};
const modelAdmission = hostServices?.modelLifecycle?.attach(options.callbacks) === true;
if (modelAdmission && hostServices?.providerText) {
hostServices.modelLifecycle?.dispose();
throw new Error(
'Remote model admission with provider-text protection requires SDK invocation context support',
);
}
const connection = remote.recovery
? connectRecoveringSdkSocket({
url: remote.url,
@ -494,11 +488,18 @@ export async function createRemoteSdkRun({
hostModel?: number;
builderModel?: number;
modelAdmission?: number;
modelAdmissionWithProviderText?: number;
};
if (capabilities?.transportVersion !== 2)
throw new Error('Worker transport capability unavailable');
if (modelAdmission && capabilities?.modelAdmission !== 1)
throw new Error('Worker model admission capability unavailable');
if (
modelAdmission &&
hostServices?.providerText &&
capabilities?.modelAdmissionWithProviderText !== 1
)
throw new Error('Worker protected model admission capability unavailable');
if (hostServices?.modelProfile && capabilities?.builderModel !== 1)
throw new Error('Worker builder model capability unavailable');
if (hostServices?.model && capabilities?.hostModel !== 1)

View file

@ -211,7 +211,7 @@ export function createWorkerModelChannel(options: {
let receiver: ReturnType<typeof createPolicyReceiver> | undefined;
let seal: ReturnType<typeof createPolicySender> | undefined;
let lifecycle: WorkerModelLifecycle | undefined;
const encodeRequest = (id: string, request: HostModelRequest) => {
const encodeRequest = (id: string, request: HostModelRequest, context?: unknown) => {
if (!receiver || !seal) throw new HostModelServiceError('incompatible');
return sealValue(
seal,
@ -219,7 +219,7 @@ export function createWorkerModelChannel(options: {
id,
publicKey: receiver.publicKey,
request,
...(lifecycle && { context: lifecycle.current() }),
...(lifecycle && { context: lifecycle.current(context) }),
},
options.limits.maxFrameBytes,
);
@ -242,12 +242,12 @@ export function createWorkerModelChannel(options: {
options.signal.addEventListener('abort', dispose, { once: true });
const service: HostModelService = {
version: 1,
invoke: async (request, signal) => {
invoke: async (request, signal, context?: unknown) => {
const id = randomUUID();
const result = reply(
await options.peer.call(
'model.invoke',
encodeRequest(id, request),
encodeRequest(id, request, context),
options.limits.maxRunMs,
signal ? AbortSignal.any([options.signal, signal]) : options.signal,
),
@ -259,9 +259,9 @@ export function createWorkerModelChannel(options: {
throw new HostModelServiceError('incompatible');
return value.message;
},
stream: async function* (request, signal) {
stream: async function* (request, signal, context?: unknown) {
const id = randomUUID();
const payload = encodeRequest(id, request);
const payload = encodeRequest(id, request, context);
const controller = new AbortController();
const boundSignal = AbortSignal.any([
options.signal,

View file

@ -30,7 +30,7 @@ import {
clearRedisTestPrefix,
} from '../../../stream/__tests__/helpers/redis';
import { createInitialModelBoundAdmissionCallback } from '../../../middleware/modelBoundContent';
import { createHostModelLifecycle } from '../admission';
import { createHostModelLifecycle, supportsModelInvocationContext } from '../admission';
import { createRedisRuntimeJournal } from './redis';
import { createRemoteSdkServer } from '../server';
import { createRuntimeRecovery } from './server';
@ -1023,9 +1023,11 @@ it('recovers a policy reply after a wire outage without repeating inspection or
}
});
it.each([false, true])(
'resumes host model chunks without repeating the provider (input admission=%s)',
async (inputAdmission) => {
it.each(['input', 'output', 'combined'] as const)(
'resumes host model chunks without repeating the provider (protection=%s)',
async (mode) => {
const inputAdmission = mode !== 'output';
const outputProtection = mode !== 'input';
const runtime = await fixture(async (input) => {
const run = await factory(input);
run.Graph!.overrideModel = input.hostModel;
@ -1048,8 +1050,8 @@ it.each([false, true])(
runtime.drop();
await delay(50);
yield new ChatGenerationChunk({
text: inputAdmission ? 'approved ' : 'PRIVATE-CANDIDATE ',
message: new AIMessageChunk(inputAdmission ? 'approved ' : 'PRIVATE-CANDIDATE '),
text: outputProtection ? 'PRIVATE-CANDIDATE ' : 'approved ',
message: new AIMessageChunk(outputProtection ? 'PRIVATE-CANDIDATE ' : 'approved '),
});
yield new ChatGenerationChunk({ text: 'third', message: new AIMessageChunk('third') });
};
@ -1101,9 +1103,13 @@ it.each([false, true])(
config: hostConfig(async () => {}),
identity,
remote: runtime.remote,
hostServices: { model, ...(inputAdmission ? { modelLifecycle } : { providerText }) },
hostServices: {
model,
...(inputAdmission && { modelLifecycle }),
...(outputProtection && { providerText }),
},
});
const content = await run.processStream(
const result = run.processStream(
{ messages: [new HumanMessage('go')] },
{
version: 'v2',
@ -1119,6 +1125,14 @@ it.each([false, true])(
: undefined,
},
);
if (mode === 'combined' && !supportsModelInvocationContext) {
await expect(result).rejects.toThrow();
expect(invocations).toBe(0);
expect(persisted).not.toHaveBeenCalled();
expect(runtime.createRun).not.toHaveBeenCalled();
return;
}
const content = await result;
expect(JSON.stringify(content)).toContain('first approved third');
expect(runtime.journalEntries.join('')).not.toContain('PRIVATE-CANDIDATE');
expect(invocations).toBe(1);

View file

@ -20,11 +20,11 @@ import type { Server } from 'node:http';
import type { RemoteSdkFactoryInput, RemoteSdkLimits, RemoteSdkStart, SdkRun } from './types';
import type { RuntimeRecoveryOptions } from './recovery/server';
import type { SdkAgentIdentity } from './identity';
import { createWorkerModelLifecycle, supportsModelInvocationContext } from './admission';
import { policyDescriptorSchema, createWorkerProviderTextPolicy } from './policy';
import { createRuntimeRecovery, recoveryClaimSchema } from './recovery/server';
import { remoteSdkLimits, isRemoteSdkSecret } from './types';
import { RECOVERY_HEADER_BYTES } from './recovery/socket';
import { createWorkerModelLifecycle } from './admission';
import { createWorkerToolService } from './hostTools';
import { RemoteSdkCheckpointer } from './checkpoint';
import { createWorkerModelChannel } from './model';
@ -298,6 +298,7 @@ export function createRemoteSdkServer(options: RemoteSdkServerOptions): {
hostModel: 1,
builderModel: 1,
modelAdmission: 1,
...(supportsModelInvocationContext && { modelAdmissionWithProviderText: 1 }),
};
if (method === 'model.chunk') return workerModel.accept(data, signal);
if (method === 'tool.result') return workerTools.accept(data);
@ -305,7 +306,7 @@ export function createRemoteSdkServer(options: RemoteSdkServerOptions): {
if (method !== 'run' || started) throw new Error('SDK execution already admitted');
started = true;
const request = startSchema.parse(data) as RemoteSdkStart;
if (request.modelAdmission && request.providerText)
if (request.modelAdmission && request.providerText && !supportsModelInvocationContext)
throw new Error(
'Combined model admission and provider-text protection is unsupported',
);