mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-10-21 17:44:11 +00:00
⏭️ feat: Honor an Interrupt Before the Model Has Answered (#15491)
* ⏭️ feat: Honor an Interrupt Before the Model Has Answered An interrupt armed while the model is still thinking sat as "Interrupting" until the entire turn finished, then landed as a terminal continuation. The cause is upstream: the SDK reads `shouldPreempt` once per streamed chunk, and its seal gate requires non-empty text — so a silent provider is never polled at all, and a reasoning-only turn is never sealable. `@librechat/agents` gains a discard-and-restart path for exactly that window (`HOOK_PREEMPT_RESTART_CAPABLE`). This wires the host half: - `GenerationJobManager.subscribePreempt` registers run-scoped wake listeners, notified after an ACCEPTED arm — local or cross-replica, since both land in `armPreemptIds`. Fenced on `createdAt` like every other preempt entry point, and a throwing listener never fails the arm a queued steer depends on. - `createSteerPreemptPoll` hands that channel to the SDK as `StreamPreemption.subscribe`, gated on a new `isSteerPreemptRestartSupported` probe so an SDK that cannot act on a wake is never handed one. The wake is a hint only: `isPreemptRequested` stays the single authority, so the level-triggered contract the seal path depends on is unchanged. Inert until `@librechat/agents` publishes the restart contract — the probe returns false against the pinned 3.7.11 and the run keeps today's behavior. * 🩹 fix: Replay an Already-Armed Preempt on Subscribe Preempt requests are level-triggered, and the window between a job becoming steerable and the SDK installing its model-attempt listener is real. An interrupt landing there was recorded with no callbacks to notify, and on a silent or reasoning-only turn no later chunk poll may ever run — leaving the request to wait out the whole turn, the exact stall this channel removes. `subscribePreempt` now replays an existing arm to the newly registered listener only, after registration: waking the whole set would re-notify runs that already looked, and waking before registration would leave a concurrent arm with nowhere to land. The `createdAt` fence is unchanged and still gates the replay. The SDK also reads the level flag once at each attempt's start, so this is belt-and-braces rather than the sole cover — but it makes the host correct on its own terms instead of resting on when the SDK happens to look. * 🔧 chore: Update @librechat/agents dependency to version 3.7.14 in package-lock.json and related package.json files * 🔧 chore: Update dependencies in package-lock.json and package.json - Bump versions for @humanfs/core (0.19.1 to 0.19.2), @humanfs/node (0.16.6 to 0.16.8), @xmldom/xmldom (0.8.13 to 0.8.15), and qs (6.15.2 to 6.16.0). - Add @humanfs/types (0.15.0) as a new dependency for @humanfs/node. - Update dependencies for side-channel and side-channel-list to their latest versions.
This commit is contained in:
parent
0219513115
commit
29f738dea8
9 changed files with 332 additions and 43 deletions
|
|
@ -640,7 +640,7 @@ class AgentClient extends BaseClient {
|
|||
hook: createSteerDrainHook(drainOptions),
|
||||
...(isSteerPreemptSupported() && {
|
||||
preemptHook: createSteerPreemptBoundaryHook(drainOptions),
|
||||
preemption: createSteerPreemptPoll(streamId),
|
||||
preemption: createSteerPreemptPoll(streamId, this.jobCreatedAt),
|
||||
}),
|
||||
...(isSteerTerminalContinuationSupported() && {
|
||||
terminalHook: createSteerTerminalContinuationHook(drainOptions),
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue