mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-01 03:26:18 +00:00
2d794a0439
* fix: Detect silently dead Redis sockets with deadline-bounded heartbeats A Redis peer can vanish without a FIN or RST (a dropped NAT entry, a proxy failover, a migrated VM). The socket stays `ready` from ioredis's point of view, no error fires, and every command on it waits for the kernel's retransmission timeout — about fifteen minutes at Linux defaults. Observed in production: while the main client's socket was dead, POST /api/agents hung until Cloudflare returned a 524; while the subscriber's socket was dead, every attach failed with "Timed out synchronizing Redis subscription" for sixteen minutes across five conversations. `REDIS_PING_INTERVAL` could not help — it awaits ping() with no deadline, so the probe just joins everything else that is waiting. - Add startRedisHeartbeat: races a PING against a deadline and destroys the socket when the probe goes unanswered, so ioredis reconnects, replays queued commands, and re-subscribes a subscriber's channels. - Destroy the stream rather than disconnect(true), whose FIN a vanished peer never acknowledges; cluster clients fall back to the regular reconnect. - Add createIoRedisSubscriber so every dedicated subscriber gets an error listener and its own heartbeat, and route the generation-stream and both subagent-routing subscribers through it. - Set ioredis keepAlive from REDIS_KEEP_ALIVE (default 10s). - Add REDIS_PING_TIMEOUT, REDIS_SUBSCRIBER_PING_INTERVAL and REDIS_KEEP_ALIVE, documented in .env.example. - Give the fake RESP server a silent mode and a SUBSCRIBE reply so the dead-peer case can be tested against real clients. * fix: Probe each cluster node and refuse a non-positive heartbeat deadline A cluster-level PING is routed to one node, so a healthy reply proved nothing about the other sockets. The heartbeat now resolves its targets each tick — every node for a cluster, the client itself otherwise — and races an independent deadline per target, tearing down only the socket that stops answering. A non-positive deadline disables the heartbeat with a warning instead of forcing a reconnect on every tick, and the single-probe test no longer depends on wall-clock slack. |
||
|---|---|---|
| .. | ||
| __tests__ | ||
| Agents | ||
| Artifacts | ||
| Config | ||
| Endpoints | ||
| Files | ||
| Runs | ||
| Schedules | ||
| Skills | ||
| start | ||
| Threads | ||
| Tools | ||
| ActionService.js | ||
| ActionService.spec.js | ||
| AssistantService.js | ||
| AuthService.js | ||
| AuthService.spec.js | ||
| cleanup.js | ||
| createRunBody.js | ||
| createRunBody.spec.js | ||
| GraphApiService.js | ||
| GraphApiService.spec.js | ||
| GraphTokenService.js | ||
| initializeMCPs.js | ||
| initializeMCPs.plugins.spec.js | ||
| initializeMCPs.spec.js | ||
| initializeOAuthReconnectManager.js | ||
| MCP.js | ||
| MCP.spec.js | ||
| MCPRequestContext.js | ||
| OboPolicyService.js | ||
| OboTokenService.js | ||
| OboTokenService.spec.js | ||
| OpenIDRefreshFlight.js | ||
| OpenIDRefreshFlight.spec.js | ||
| OpenIDRefreshRecovery.js | ||
| OpenIDSessionRefresh.js | ||
| OpenIDSessionRefresh.spec.js | ||
| PermissionService.js | ||
| PermissionService.spec.js | ||
| PluginService.js | ||
| PluginService.spec.js | ||
| RefreshTokenBridge.js | ||
| RefreshTokenBridge.spec.js | ||
| systemGrant.spec.js | ||
| ToolService.js | ||
| twoFactorService.js | ||
| twoFactorService.spec.js | ||