🛫 refactor: Promote Generation Protocol V2 Automatically (#15324)

* refactor: promote generation protocol v2 automatically

* fix: remove unused protocol import

.env.example

@@ -1165,13 +1165,6 @@ HELP_AND_FAQ_URL=https://librechat.ai
 # Emergency global stop for both automatic and manual scheduled runs.
 # SCHEDULES_DISABLED=true
 
-# Generation stream wire/state protocol. Redis-backed deployments default to the
-# rolling-upgrade-safe v1 protocol when this is unset; in-memory deployments use v2.
-# After EVERY replica is running a v2-capable LibreChat build and all v1-owned active
-# generations have drained, set this to 2 in a second rollout. Do not roll back to a
-# pre-v2 build while v2 generations remain active in Redis.
-# GENERATION_PROTOCOL_VERSION=2
-
 # Coalesce streamed model/tool-argument deltas into windowed Redis publications (ms).
 # Unset or 0 (default) publishes per delta. 25 is recommended: it batches the publish
 # EVAL and the durable append across the window (fewer Redis round trips and lower

CONTEXT.md

@@ -12,7 +12,7 @@
 - **Agent event handling outcome**: the durable, generation-fenced result of a previously accepted event delivery. `started` proves generation admission; terminal states distinguish verified tool application, clean completion without action, failure, and cancellation. Transport success remains separate so an accepted event cannot masquerade as completed work.
 - **Agent event expected action**: an optional source-declared tool name and bounded argument subset evaluated against host-observed completed run steps. It is evidence policy, not authorization and not a model-authored success claim.
 - **Event actor head**: the private, durable pointer on an event-bound child conversation to its latest committed LangGraph checkpoint, plus one previous checkpoint for safe cleanup. Only a qualifying applied action advances it through compare-and-swap; failed, cancelled, or no-action invocations leave it unchanged. A legacy-path event marks the head for a cold rebuild from durable message history before fork mode can resume. Every applied commit conflict, unverified commit, or post-commit persistence failure is retained in a private reconciliation journal that blocks later actor turns instead of continuing from stale state; an exact marker can be cleared only after its checkpoint is verified authoritative, its history is repaired, or its external action is explicitly compensated.
-- **Event actor invocation fork**: a delivery-owned checkpoint namespace copied from the event actor head. A warm invocation receives only the new trusted event, then commits its terminal checkpoint when the expected action is observed or deletes the fork otherwise. When the invocation pauses for approval or Ask User, the SDK emits signed, versioned suspension evidence. The child Conversation is the canonical one-shot suspension authority; the generation job carries only a versioned projection for UI, rolling-deploy routing, and the existing resume endpoint. Shared-store deployments publish new suspensions only under generation protocol v2, after the homogeneous-fleet cutover; protocol v1 keeps pause-capable actors on the legacy history path so an old resume consumer cannot consume unknown evidence. A resume shares one identity between its Conversation claim and provider-owner CAS, clears the predecessor projection, and publishes a successor only after a re-pause is canonical. A pending interrupt takes precedence over expected-action evidence from the same segment; if that segment already applied the expected action, publishing its successor pause cold-marks the prior head until a later applied commit replaces it. An ambiguous projection write is accepted only after reading back the exact generation, action, and suspension. The provider-start CAS is written only after client reconstruction and immediately before the continuation gate opens, then retains its exact execution identity after drain, so terminal recovery can compensate a projected claim only when that identity proves execution never began. Durable approval projection is exposed before the persistence barrier opens, preventing a resolved action from being announced afterward. Terminal no-action retirement cancels or settles the exact suspension and releases its delivery-side action admission before public settlement; if retention already removed the child Conversation, the delivery remains authoritative for its exact admission identity. Resume, re-pause, cancellation, and expiry claim or replace that exact suspension before touching its job projection, so later mailbox deliveries stay blocked until terminal history and handling evidence settle.
+- **Event actor invocation fork**: a delivery-owned checkpoint namespace copied from the event actor head. A warm invocation receives only the new trusted event, then commits its terminal checkpoint when the expected action is observed or deletes the fork otherwise. When the invocation pauses for approval or Ask User, the SDK emits signed, versioned suspension evidence. The child Conversation is the canonical one-shot suspension authority; the generation job carries only a versioned projection for UI, rolling-deploy routing, and the existing resume endpoint. Current Event Actor hosts select generation protocol v2 automatically; immutable protocol-v1 jobs remain consumable through the legacy history path while they drain. A resume shares one identity between its Conversation claim and provider-owner CAS, clears the predecessor projection, and publishes a successor only after a re-pause is canonical. A pending interrupt takes precedence over expected-action evidence from the same segment; if that segment already applied the expected action, publishing its successor pause cold-marks the prior head until a later applied commit replaces it. An ambiguous projection write is accepted only after reading back the exact generation, action, and suspension. The provider-start CAS is written only after client reconstruction and immediately before the continuation gate opens, then retains its exact execution identity after drain, so terminal recovery can compensate a projected claim only when that identity proves execution never began. Durable approval projection is exposed before the persistence barrier opens, preventing a resolved action from being announced afterward. Terminal no-action retirement cancels or settles the exact suspension and releases its delivery-side action admission before public settlement; if retention already removed the child Conversation, the delivery remains authoritative for its exact admission identity. Resume, re-pause, cancellation, and expiry claim or replace that exact suspension before touching its job projection, so later mailbox deliveries stay blocked until terminal history and handling evidence settle.
 - **Event actor receipt**: the private, terminal proof stored on the authoritative `AgentTriggerDelivery` row for one bound actor invocation. Its unique delivery identity, terminal resolution, exact checkpoint, and bounded action identity provide replay and recovery for the retention window without storing prompts, events, tool arguments, tool output, or conversation history. It does not own the actor checkpoint; the conversation keeps only the actor head and any active unresolved reconciliation until this receipt is durable.
 - **Agent event actor mailbox**: the automatic durable delivery-ordering lane for one authenticated source binding. It keeps later deliveries queued after transport admission until the current child turn records an authoritative terminal handling outcome. It serializes existing coalesced batches and individual events without becoming a second execution controller or actor checkpoint store.
 - **Agent trigger capability shield**: the durable mixed-version representation for internal trigger work that only a capability-aware worker may execute. Mongo uses an old-publishable `staging` shell; a queued `leased` shell without an owner or deadline, which old workers cannot claim but can use for bounded lane rechecks; a private lease only during execution; and a legacy-terminal `capability_dead` shell once dead. Private capability fields own current claiming, retry, and dead-letter truth. Redis uses a versioned fail-closed terminal status and recovery index that old replacement scripts and sweepers cannot consume. The shield is an implementation detail at the storage seam, never a deployment switch or user-configured product mode.

api/server/controllers/agents/__tests__/protocol.spec.js

@@ -8,16 +8,6 @@ const {
 } = require('../protocol');
 
 describe('generation protocol negotiation', () => {
-  const configuredProtocol = process.env.GENERATION_PROTOCOL_VERSION;
-
-  afterEach(() => {
-    if (configuredProtocol == null) {
-      delete process.env.GENERATION_PROTOCOL_VERSION;
-    } else {
-      process.env.GENERATION_PROTOCOL_VERSION = configuredProtocol;
-    }
-  });
-
   test('missing, invalid, or conflicting advertisements fail closed to v1', () => {
     expect(getRequestedGenerationProtocol({})).toBe(GENERATION_PROTOCOL_V1);
     expect(
@@ -44,31 +34,20 @@ describe('generation protocol negotiation', () => {
     ).toBe(GENERATION_PROTOCOL_V2);
   });
 
-  test('defaults Redis to the bridge protocol and in-memory storage to v2', () => {
-    delete process.env.GENERATION_PROTOCOL_VERSION;
-    expect(getServerGenerationProtocol({ isRedis: true })).toBe(GENERATION_PROTOCOL_V1);
-    expect(getServerGenerationProtocol({ isRedis: false })).toBe(GENERATION_PROTOCOL_V2);
+  test('advertises protocol v2 for every built-in generation store', () => {
+    expect(getServerGenerationProtocol()).toBe(GENERATION_PROTOCOL_V2);
   });
 
-  test('honors an explicit fleet-wide protocol gate', () => {
-    process.env.GENERATION_PROTOCOL_VERSION = '2';
-    expect(getServerGenerationProtocol({ isRedis: true })).toBe(GENERATION_PROTOCOL_V2);
-    process.env.GENERATION_PROTOCOL_VERSION = '1';
-    expect(getServerGenerationProtocol({ isRedis: false })).toBe(GENERATION_PROTOCOL_V1);
-  });
-
-  test('negotiates new jobs against the server ceiling', () => {
-    delete process.env.GENERATION_PROTOCOL_VERSION;
-    const req = {
+  test('selects the protocol advertised by a new-generation client', () => {
+    const current = {
       body: { generationProtocolVersion: 2 },
       headers: { 'x-librechat-generation-protocol': '2' },
     };
-    expect(negotiateNewGenerationProtocol(req, { isRedis: true })).toBe(GENERATION_PROTOCOL_V1);
-    expect(negotiateNewGenerationProtocol(req, { isRedis: false })).toBe(GENERATION_PROTOCOL_V2);
+    expect(negotiateNewGenerationProtocol(current)).toBe(GENERATION_PROTOCOL_V2);
+    expect(negotiateNewGenerationProtocol({})).toBe(GENERATION_PROTOCOL_V1);
   });
 
-  test('never upgrades a live v1 job after the global gate flips', () => {
-    process.env.GENERATION_PROTOCOL_VERSION = '2';
+  test('never upgrades a live v1 job after new generations move to v2', () => {
     const req = {
       query: { generationProtocolVersion: '2' },
       headers: { 'x-librechat-generation-protocol': '2' },

api/server/controllers/agents/__tests__/steer.spec.js

@@ -213,29 +213,19 @@ describe('SteerController (wrapper)', () => {
     expect(res.headers[GENERATION_PROTOCOL_HEADER]).toBe('1');
   });
 
-  it('honors a v1 server rollout gate even when the request advertises exact v2', async () => {
-    const previous = process.env.GENERATION_PROTOCOL_VERSION;
-    process.env.GENERATION_PROTOCOL_VERSION = '1';
+  it('passes an exact v2 advertisement to the package host contract', async () => {
     mockHandleSteerRequest.mockResolvedValue({
       status: 202,
-      body: { status: 'queued', generationProtocolVersion: 1 },
+      body: { status: 'queued', generationProtocolVersion: 2 },
     });
-    try {
-      await request(buildApp())
-        .post('/chat/steer')
-        .set('X-LibreChat-Generation-Protocol', '2')
-        .send({ conversationId: 'c1', text: 'hello', generationProtocolVersion: 2 });
+    await request(buildApp())
+      .post('/chat/steer')
+      .set('X-LibreChat-Generation-Protocol', '2')
+      .send({ conversationId: 'c1', text: 'hello', generationProtocolVersion: 2 });
 
-      expect(mockHandleSteerRequest.mock.calls[0][2]).toEqual(
-        expect.objectContaining({ generationProtocolVersion: 1 }),
-      );
-    } finally {
-      if (previous == null) {
-        delete process.env.GENERATION_PROTOCOL_VERSION;
-      } else {
-        process.env.GENERATION_PROTOCOL_VERSION = previous;
-      }
-    }
+    expect(mockHandleSteerRequest.mock.calls[0][2]).toEqual(
+      expect.objectContaining({ generationProtocolVersion: 2 }),
+    );
   });
 
   it('uses the package job cap, not the host maximum, for the final response marker', async () => {

api/server/controllers/agents/protocol.js

@@ -34,18 +34,10 @@ function getRequestedGenerationProtocol(req) {
     : GENERATION_PROTOCOL_V1;
 }
 
-/**
- * Redis state is shared by every replica, including an older binary during a
- * rolling deployment. Default it to the bridge-safe v1 protocol until an
- * operator completes the documented homogeneous-fleet cutover. In-memory
- * state cannot be touched by another process, so it can use v2 immediately.
- */
-function getServerGenerationProtocol(manager) {
-  const configured = parseProtocolVersion(process.env.GENERATION_PROTOCOL_VERSION);
-  if (configured != null) {
-    return configured;
-  }
-  return manager?.isRedis === true ? GENERATION_PROTOCOL_V1 : GENERATION_PROTOCOL_V2;
+/** The current server contract supports protocol v2 for every built-in store.
+ * Client advertisement still decides whether a new generation uses v1 or v2. */
+function getServerGenerationProtocol() {
+  return GENERATION_PROTOCOL_V2;
 }
 
 function getJobGenerationProtocol(job) {
@@ -53,8 +45,8 @@ function getJobGenerationProtocol(job) {
 }
 
 /** Selects an immutable protocol for a newly created generation. */
-function negotiateNewGenerationProtocol(req, manager) {
-  return Math.min(getRequestedGenerationProtocol(req), getServerGenerationProtocol(manager));
+function negotiateNewGenerationProtocol(req) {
+  return Math.min(getRequestedGenerationProtocol(req), getServerGenerationProtocol());
 }
 
 /**

api/server/controllers/agents/request.js

@@ -573,7 +573,7 @@ function rejectMissingTriggerParentMessageId(res, generationProtocolVersion) {
  */
 const ResumableAgentController = async (req, res, next, initializeClient, addTitle) => {
   const startupTelemetry = getAgentStartupTelemetry(req);
-  let generationProtocolVersion = negotiateNewGenerationProtocol(req, GenerationJobManager);
+  let generationProtocolVersion = negotiateNewGenerationProtocol(req);
   const {
     text,
     isRegenerate,

api/server/controllers/agents/resume.js

@@ -691,7 +691,7 @@ async function finalizeResumedTurn({
  */
 const ResumeAgentController = async (req, res, next, initializeClient, addTitle) => {
   const userId = req.user.id;
-  let generationProtocolVersion = negotiateNewGenerationProtocol(req, GenerationJobManager);
+  let generationProtocolVersion = negotiateNewGenerationProtocol(req);
   const { conversationId, actionId, generationCreatedAt } = req.body;
   const streamId = conversationId;
 

api/server/controllers/agents/steer.js

@@ -1,6 +1,5 @@
 const {
   checkAccess,
-  GenerationJobManager,
   handleSteerRequest,
   handleSteerCancel,
   handleSteerArm,
@@ -25,7 +24,7 @@ const db = require('~/models');
 
 /** Upper bound before the package reads the immutable live-job marker. */
 const getHostGenerationProtocol = (req) =>
-  Math.min(getRequestedGenerationProtocol(req), getServerGenerationProtocol(GenerationJobManager));
+  Math.min(getRequestedGenerationProtocol(req), getServerGenerationProtocol());
 
 /** The package returns its job-capped effective marker in every body. Keep the
  * header and JSON inseparable at this final serialization boundary. */

api/server/routes/agents/index.js

@@ -64,10 +64,7 @@ function hasTenantMismatch(job, user) {
  * validation, not-found, and authorization envelopes; it never leaks an
  * existing job's marker to an unauthorized caller. */
 function negotiateRequestGenerationProtocol(req) {
-  return Math.min(
-    getRequestedGenerationProtocol(req),
-    getServerGenerationProtocol(GenerationJobManager),
-  );
+  return Math.min(getRequestedGenerationProtocol(req), getServerGenerationProtocol());
 }
 
 /** Every generation-control JSON envelope carries the exact numeric protocol
@@ -94,9 +91,7 @@ async function sendJoblessStatus(req, res, conversationId) {
   );
   const generationProtocolVersion = Math.min(
     requestedProtocolVersion,
-    claimed.steers.length > 0
-      ? claimed.generationProtocolVersion
-      : getServerGenerationProtocol(GenerationJobManager),
+    claimed.steers.length > 0 ? claimed.generationProtocolVersion : getServerGenerationProtocol(),
   );
   res.set(GENERATION_PROTOCOL_HEADER, String(generationProtocolVersion));
   return res.json({

helm/librechat/readme.md

@@ -58,19 +58,13 @@ also register this LibreChat callback URL with your identity provider:
 https://<librechat-domain>/api/admin/oauth/openid/callback
 ```
 
-## Generation protocol rollout
+## Generation protocol compatibility
 
-Redis-backed generation streams use protocol v1 by default during the first
-rollout of a v2-capable image. This keeps a rolling deployment compatible with
-replicas that still run the previous Redis queue, checkpoint, and recovery
-scripts.
-
-After every LibreChat replica is on the v2-capable image and all active
-generations owned by the old image have drained, set
-`librechat.configEnv.GENERATION_PROTOCOL_VERSION="2"` in a second rollout.
-Keep the new image in place until v2 generations have drained; an older image
-cannot safely operate on their Redis state. In-memory generation streams do not
-share state across replicas and negotiate v2 without this cutover.
+Generation protocol v2 is selected automatically; no deployment setting is
+required. Rolling upgrades must start from a v2-capable bridge release
+(LibreChat `v0.8.8-rc1` or newer, or Helm chart `2.0.8` or newer). When
+upgrading from an older release, stop the old replicas before starting the new
+image so pre-v2 and automatic-v2 binaries never share generation state in Redis.
 
 ## Langfuse Fanout
 

packages/api/src/stream/__tests__/RedisJobStore.stream_integration.spec.ts

@@ -52,10 +52,6 @@ describe('RedisJobStore Integration Tests', () => {
     process.env.REDIS_KEY_PREFIX = testPrefix;
     process.env.REDIS_PING_INTERVAL = '0';
     process.env.REDIS_RETRY_MAX_ATTEMPTS = '5';
-    // This suite exercises the receipt-safe current behavior. Rollout-specific
-    // v1 defaults and mixed-client downgrade paths live in protocolRollout.
-    process.env.GENERATION_PROTOCOL_VERSION = '2';
-
     jest.resetModules();
 
     // Import Redis client

packages/api/src/stream/__tests__/protocolRollout.stream_integration.spec.ts

@@ -7,12 +7,10 @@ import { RedisJobStore } from '../implementations/RedisJobStore';
 
 describe('Redis generation protocol rollout bridge', () => {
   const keyPrefix = `Protocol-Rollout-${process.pid}-${Date.now()}:`;
-  const originalProtocol = process.env.GENERATION_PROTOCOL_VERSION;
   let redis: RedisTestClient;
   let store: RedisJobStore;
 
   beforeAll(async () => {
-    delete process.env.GENERATION_PROTOCOL_VERSION;
     redis = createRedisTestClient(keyPrefix);
     await redis.connect();
     store = new RedisJobStore(redis);
@@ -26,17 +24,12 @@ describe('Redis generation protocol rollout bridge', () => {
   afterAll(async () => {
     await store.destroy();
     await redis.quit();
-    if (originalProtocol == null) {
-      delete process.env.GENERATION_PROTOCOL_VERSION;
-    } else {
-      process.env.GENERATION_PROTOCOL_VERSION = originalProtocol;
-    }
   });
 
-  test('Redis defaults legacy jobs to v1 and isolates checkpoints only for v2', async () => {
-    const legacy = await store.createJob('redis-protocol-v1', 'user-1');
-    const current = await store.createJob('redis-protocol-v2', 'user-1', undefined, undefined, {
-      generationProtocolVersion: 2,
+  test('Redis defaults new jobs to v2 while explicit v1 remains compatible', async () => {
+    const current = await store.createJob('redis-protocol-v2', 'user-1');
+    const legacy = await store.createJob('redis-protocol-v1', 'user-1', undefined, undefined, {
+      generationProtocolVersion: 1,
     });
 
     expect(legacy.generationProtocolVersion).toBe(1);

packages/api/src/stream/implementations/RedisJobStore.ts

@@ -1907,14 +1907,12 @@ export class RedisJobStore implements IJobStoreV2 {
     }
     const safeInitialMetadata = { ...initialMetadata };
     delete safeInitialMetadata.providerDrained;
-    let generationProtocolVersion: 1 | 2 = 1;
+    let generationProtocolVersion: 1 | 2 = 2;
     if (
       initialMetadata.generationProtocolVersion === 1 ||
       initialMetadata.generationProtocolVersion === 2
     ) {
       generationProtocolVersion = initialMetadata.generationProtocolVersion;
-    } else if (process.env.GENERATION_PROTOCOL_VERSION === '2') {
-      generationProtocolVersion = 2;
     }
     const job: CreatedJobData = {
       ...safeInitialMetadata,