LibreChat/CONTEXT.md
Danny Avila 2ffee7e143
🪂 feat: Opt-In Text Fallback for Files No Tool Can Read (#15940)
* 🪂 feat: Opt-In Text Fallback for Files No Tool Can Read

An upload routed to tools (`llmDeliveryPath: none`) reaches nothing when the
agent handling the turn runs neither Run Code nor File Search, or runs one that
cannot read the type. With `fileConfig.endpoints.<endpoint>.textFallbackWithoutTools`
(also accepted at the top level and inherited from `default`), such a turn
delivers the text extracted at upload instead. The setting is off by default,
which keeps today's behavior.

- Upload: when the setting is on and an inferred `none` route finds no reading
  tool, the built-in document parser or native text reader stores the text on
  the file, best effort, under the extracted-text size cap and content policy.
- Turn: `resolveTurnLLMDeliveryPath` re-resolves each inferred route against
  the tools the agent runs, and `applyTurnTextFallback` marks copies of the
  records a turn loads, so endpoint filtering, limits, inspection, history
  replay and steer media all see the text the turn delivers. The stored route
  is never rewritten.
- Explicit destinations, records predating routing, and turns whose tools are
  unknown are left as they are.

* 🧪 test: Resolve Custom Endpoint Fallback Under the Endpoint Name

Uploads route a custom endpoint agent by its saved provider, the endpoint name. initializeAgent marks turn copies before getProviderConfig swaps the provider for the backing client, and the endpoint keeps the name afterward, so the client and child encoders resolve the opt-in under the same endpoint. Cover both the initialization path and the post-initialization agent shape.

* ♻️ refactor: Share One Extracted-Text Storage Cap Across Uploads

The fallback defined its own copy of the 15 MiB cap context uploads already apply to extracted text. Export MAX_STORED_EXTRACTED_TEXT_BYTES from files/extract and use it in both places, documenting that it bounds what a MongoDB file document can hold rather than operator policy.

* 🧭 fix: Keep Fallback Text Wherever a Later Turn Can Deliver It

Extraction skipped uploads filed under a reading tool, so a handoff agent
without that tool, or the same agent after its tools or grants changed, had
no text to fall back to. Store fallback text for every inferred route on a
message attachment; files kept on an agent's tool resources never reach a
prompt and still skip it.

Turn marking only ran while the fallback was enabled, so a stored tool-routed
record whose type the endpoint now routes to text stayed marked `none` and
`extractFileContext` skipped it. applyTurnTextDelivery (renamed from
applyTurnTextFallback) now marks any stored `none` record with text that the
turn resolves to text, and returns early when no such record is loaded.

* 🚦 fix: Admit Every Attachment by the Route Its Turn Delivers

A record upload routed to tools that the current endpoint sends to the
provider stayed marked `none` on the turn copy, so admission skipped it while
BaseClient encoded its bytes, bypassing attachment count and size limits.
applyTurnDelivery (renamed from applyTurnTextDelivery) now gives every record
whose route upload inferred the route the turn resolves, before endpoint
filtering, limits and inspection, matching the run-file encoder.
hasInferredLLMDeliveryPath shares the inferred-route rule with the resolver.

Fallback extraction selected the document parser inside the CJS upload path.
resolveUploadFallbackText now takes the upload and picks parseDocument or
parseTextNative itself, with the extractors injectable, so process.js passes
request data only. Its tests run the real parser against the xlsx fixture and
the real native reader against temporary files.

* 🎚️ fix: Read Custom Endpoint Dialect From Config on Both Sides of Init

initializeAgent materializes turn routes before getProviderConfig swaps the
saved custom endpoint name in agent.provider for its backing client, so
routing read that name as a non-OpenAI dialect. Media a custom endpoint opted
into could then be materialized off the provider path, skipping admission,
while the finalized client still sent the bytes. resolveAgentDeliveryRouting
now reads the dialect with getCustomEndpointProvider, as upload does, which
holds before and after the swap.

The upload fallback plan now keys on the chosen-destination marker the turn
resolver reads, so an upload recorded as chosen, including every legacy
chooser upload, never pays for text a turn cannot deliver.

* 🛡️ fix: Apply Turn Routes Only While the Record Stays Admitted

Resolution while initializing and at delivery can disagree: delivery reads a
Responses API choice only the finished client config holds. Applying a route
that takes a record out of admission, off the model path or to text it never
stored, would then skip the limits for a file the client still sends.
applyTurnDelivery now applies a route only while the record stays model-bound;
admitting a record the turn leaves out cannot slip past a limit.

* 🕹️ feat: Run Declared Project Actions on Attached Workers (#15943)

* feat: use declared attached project actions

* Complete named action definitions and approval previews

* Sort named environment imports

* Sort native environment fixture imports

* 🔓 fix: Keep Code Approval Mode Selectable Mid-Run (#15938)

* 🔓 fix: Keep Code Approval Mode Selectable Mid-Run

The composer's code approval mode selector was disabled while a run streamed, and a mode picked during a run was reverted when the final or abort event merged the server conversation back. The selector now stays usable in flight and both merges retain a mid-run selection, which the next send carries.

* 🔓 fix: Retain Mid-Run Approval Mode Across Cache and Recovery

Stamp the live approval mode onto the submission conversation at send so the retention baseline matches what was read, keep a retained mode in the conversation query cache on final and cancel, and rebuild failed or aborted conversations from a preset that carries the mid-run pick.

* 🔓 fix: Scope Retained Approval Mode to the Submitted Conversation

Retention now requires the live conversation to be the one the run submitted, or the id the server assigned to a new chat, so navigating elsewhere mid-run cannot write that conversation's mode back. A single recovery helper rebuilds failed or aborted conversations from the retained preset and patches the detail cache for every recovery site.

* 🔓 refactor: Keep the Local Approval Mode on Every Server Merge

Replace per-path retention with one rule: a locally picked code approval mode is newer than any server copy of the same conversation, so the final and abort merges and the error recovery preset keep it. The selector writes the pick into the conversation's detail cache, which navigation rebuilds from, so the pick is conversation-scoped without reading the index-global atom from the SSE handlers.

* 🔓 fix: Prefer the Live Conversation When Caching the Final Merge

The final handler's detail-cache merge now takes the local approval mode from the open conversation when it is the one that finished, falling back to the cached record only for a conversation that is no longer on screen.

* 🔓 fix: Keep the Local Approval Mode Through Settled-Start Reconciliation

The resumable transport's settled-start reconciliation and replacement handoff wrote a fetched conversation straight into conversation state and the detail cache. Both now apply the same local-mode rule as the event handlers, treating a new chat's pending id as the settled conversation's own.

* 🔓 fix: Seed the Detail Record When a Mode Is Picked

A conversation that has its id but no detail record yet now gets one from the live conversation when a mode is picked, the same key the resumable transport seeds optimistically, so recovery and navigation find the pick. A chat with no id yet still keeps the pick in conversation state alone.

* ⬆️ chore: bump agents to 3.8.7 (#15947)

* 🧭 refactor: Settle Turn Delivery Routing Once in Agent Initialization

`initializeAgent` resolved the provider and its client options only after the
turn's files were loaded and admitted, and the two delivery readers each rebuilt
the attachment routing from the agent object at their own moment: `BaseClient`
took the custom-endpoint dialect from the already-swapped `agent.provider`, the
run-file encoder from whatever child config it was handed.

Move `getProviderConfig`/`getOptions` ahead of file discovery, where nothing in
between fed them, and settle one `deliveryRouting` value with every input final:
the file policy under the endpoint's own name, the dialect its config declares,
the Responses API decision the model call uses, and the transcription setting.
`InitializedAgent`, the child encoder and `BaseClient` consume that value;
`resolveTurnLLMDeliveryPath` is the one place a stored route is resolved again.

* fix: Preserve attachment reachability across handoffs and late steers

* test: Type the untrusted attachment reference fixture

---------

Co-authored-by: Ravi Kumar L <upman@users.noreply.github.com>
2026-09-14 21:02:54 -04:00

17 KiB

Domain language

  • Scheduled run admission: The claimed-occurrence phase that rehydrates the owner, validates current schedule policy and agent reachability, resolves files and MCP readiness, and only then competes for durable generation capacity. It owns cancellation and lease revalidation until a generation slot is reserved; a slow or failed readiness check never occupies generation capacity.

  • Attached code environment: A principal- or deployment-authorized stateful workspace owned by an outbound librechat-code worker on a user-chosen machine or VM. LibreChat selects it and enforces approval policy, Code API authenticates and dispatches to it, and the worker's local sandbox and capability flags remain the final execution ceiling. The environment interface is runtime-neutral: native SRT, WSL2, Docker/NsJail, and future adapters expose the same workspace operations without leaking host paths or runtime configuration into agent tools.

  • Conversation code-environment decision: The immutable choice established by a conversation's first accepted submission between validated attached workspaces and continuing without an attached environment. Agent defaults and recent workspace preferences may suggest a draft choice, but only the persisted conversation decision authorizes attached workspace tool registration; later turns, retries, resumes, and alternate ingresses cannot upgrade or replace it. Runs persist a decision only for a conversation that does not store one yet, so no run writes its run-start decision back over a stored one. The one exception to immutability is the owner's explicit move, available where statefulCodeSessions.conversationMoves.enabled allows it: when an agent is pointed at a different attached environment after the chat was sealed, the owner may replace the decision's environments with the ones the agents now use, validated against the live worker. A move never changes the workspace of an environment the decision already covers, never upgrades a conversation that continues without an attached environment, and is refused while a generation is running, awaiting approval, or saving its response.

  • Agent run envelope: the versioned, JSON-safe request contract created after ingress authentication and protocol validation but before agent, provider, tool, or MCP initialization. It carries only the validated protocol payload and the minimum trusted principal identifiers. The execution host rehydrates all runtime state from those identifiers.

  • Agent execution context: runtime-only, transport-free state rehydrated beside an Agent run envelope. It contains the authenticated user, application configuration, normalized request metadata, and resolved conversation facts needed by initialization, but never Express request/response objects or serialized credentials.

  • Agent execution host: the protocol-neutral module that owns run admission, disconnect cancellation, provider-start fencing, and terminal settlement. Protocol implementations execute behind its callback interface; HTTP adapters retain validation and final stream rendering.

  • Agent execution enrollment: the durable, protocol-neutral lifecycle authority for an admitted Agent run. It is created under the authenticated user and tenant before user-owned initialization, rechecks the shared owner-deletion admission fence after registration, exposes the only provider abort signal, fences exact provider start, terminalizes the run, waits for every trailing usage, artifact, and stored-response write, and acknowledges provider drain last. A transient terminalization failure is reconciled after trailing writes; provider drain is never acknowledged while the exact job remains nonterminal. Delete-all holds the owner fence, drains every owner run before selecting its first persistence snapshot, and repeats both the drain and an idempotent owner-persistence sweep after any recovered fence lapse before releasing admission. Exact-conversation deletion additionally performs an unconditional idempotent cleanup over its immutable deleted-ID set because a fully drained run may leave the active index after racing the first delete; only the explicit empty result is benign, while storage failures remain fatal. Chat Completions, Responses, Channels, and future ingress adapters share this authority without moving LibreChat persistence policy into the Agents SDK.

  • Agent turn execution plan: the immutable, request-local decision compiled once after authentication, agent resolution, and tool initialization. It records the trusted turn origin, conversation lineage, pause capability, binding/action context, and the preferred checkpoint, history, or fresh state-loading strategy without executing the model or owning persistence. Checkpoint failure falls back to durable history within the same Agents lifecycle.

  • Turn delivery routing: the per-agent, request-local value that decides how each attachment reaches the model on one turn (provider, text, or none). Initialization settles it once, after the provider swap and the Responses API decision, under the endpoint's own name and the media dialect its config declares. Every reader of a turn route consumes that one value rather than deriving it from the agent. A stored route is an upload-time inference that this value resolves again for the turn; a destination the user chose stands.

  • Effective agent selection: the resolved endpoint and agent identity after an enforced model spec is applied. Authorization and agent loading must consume this same identity before the Agent run envelope is initialized.

  • MCP runtime request body: trusted chat identifiers supplied only while an MCP server handles an agent request. It enables request-scoped header placeholders without retaining user-specific request data on a shared server definition.

  • MCP direct OpenID bearer: an operator-trusted remote MCP credential mode that resolves the logged-in user's live OpenID access token into an Authorization header. It may replace one rejected connection after a forced session refresh, but it never replays the rejected tool invocation automatically.

  • MCP OAuth prompt projection: the client-safe, generation-scoped view of authorization prompts that remain actionable in a resumable Agent stream. It is derived from durable OAuth step state, carries no tokens or flow internals, and lets reconnecting clients render current state without interpreting the replay log.

  • Caller Capability Projection: the versioned, SDK-owned classification of currently active tools by direct and programmatic callers. Event-driven execution transports this projection as data; LibreChat intersects it with its trusted registry and never recomputes deferred-tool discovery policy or treats the projection as authorization.

  • Subagent thread: a durable, view-only child conversation owned by one parent conversation and subagent identity. A parent agent may continue it by stable threadId; each continuation uses a fresh execution lease restored from the canonical child transcript. It is not an ordinary human-writable chat.

  • Live subagent task owner: the one API process holding a detached child execution, its abort controller, and its bounded control queue. Redis may route trusted poll/control envelopes to that owner, but it does not migrate or persist the executor; Mongo persists only the logical child thread and its continuation fence.

  • Subagent completion wakeup: a durable internal continue trigger pre-registered before detached child execution so a process crash cannot lose the wakeup. Delivery defers until the child's terminal transcript is persisted, targets the initiating agent and exact parent response branch, carries task metadata rather than child output, waits for the parent generation to settle, and starts the parent turn that collects the result through the existing task store.

  • Agent continuation preparation: the single source-dispatch seam that resolves a durable continue delivery immediately before admission. Bound Event Actor work selects its binding adapter; internal completion work selects an adapter by stable source identity. Preparation may resolve authoritative input and branch state or settle already-consumed work, but it does not own source result truth, delivery ordering, or generation execution.

  • Warm terminal steer continuation: a queued steer accepted before a generation's terminal boundary may continue the same SDK Run without creating a replacement generation. After parallel Stop hooks fold, the serialized StopFinalize phase tells the job store whether another continuation is already planned or terminal progress is forbidden. The store atomically chooses among claiming the current protocol-v2 FIFO batch, keeping empty admission open for an already-planned segment, and sealing admission so every racing or later message becomes an ordinary follow-up. Claimed steer receipts remain the crash-recovery authority; protocol-v1 generations always seal because they cannot recover an ambiguous terminal claim. Tool-batch, preemption, and terminal boundaries share one durable apply-and-inject adapter, while the SDK owns the bounded Stop-continuation loop.

  • Agent queued turn: a server-owned ordinary follow-up accepted while an Agent generation is active. Its Mongo row is the sole FIFO, payload, and lifecycle authority; the trigger delivery is only a replayable wakeup and Agent execution enrollment is only an execution adapter. The lifecycle reserves the deterministic delivery identity before publication, admits only after the captured branch has a clean durable predecessor outcome, and commits the source-owned generation receipt only after provider invocation has been enrolled. An accepted or deduplicated loopback response requires that exact receipt; a process death before it remains explicit admission-indeterminate evidence instead of silently consuming the text. Admission reconciliation is leased, backoff-scheduled work: exact generation evidence may repair legacy ambiguity, while current source-owned ambiguity never infers provider outcome from transient job state. Aborted or failed predecessors and exhausted admission attempts remain visible terminal rows. Transport-ambiguous enqueue outcomes stay non-resendable until exact request-identity reconciliation succeeds; an elapsed client observation window may hide the warning locally but never converts ambiguity into permission to submit again. Conversation deletion cancels the rows, retires their deliveries, and removes their payloads before deleting the conversation wave.

  • Subagent activity stream: an observational, task-scoped live projection of bounded child progress for the currently open private panel. It may cross API replicas through Redis, never carries hidden reasoning text, and never controls or settles execution. The durable child thread remains canonical and its existing polling view is the fallback for missed or unavailable live events.

  • Agent event handling outcome: the durable, generation-fenced result of a previously accepted event delivery. started proves generation admission; terminal states distinguish verified tool application, clean completion without action, failure, and cancellation. Transport success remains separate so an accepted event cannot masquerade as completed work.

  • Agent event expected action: an optional source-declared tool name and bounded argument subset evaluated against host-observed completed run steps. It is evidence policy, not authorization and not a model-authored success claim.

  • Event actor head: the private, durable pointer on an event-bound child conversation to its latest committed LangGraph checkpoint, plus one previous checkpoint for safe cleanup. Only a qualifying applied action advances it through compare-and-swap; failed, cancelled, or no-action invocations leave it unchanged. A legacy-path event marks the head for a cold rebuild from durable message history before fork mode can resume. Every applied commit conflict, unverified commit, or post-commit persistence failure is retained in a private reconciliation journal that blocks later actor turns instead of continuing from stale state; an exact marker can be cleared only after its checkpoint is verified authoritative, its history is repaired, or its external action is explicitly compensated.

  • Event actor invocation fork: a delivery-owned checkpoint namespace copied from the event actor head. Every bound Event Actor enters one turn module; fresh, history, and checkpoint are internal state-loading adapters selected automatically from actor state and immutable request capability, never from operator configuration. The history adapter owns its durable turn fence and token ordering, while immutable protocol-v1 tokens remain read-compatible until their jobs drain. A warm invocation receives only the new trusted event, then commits its terminal checkpoint when the expected action is observed or deletes the fork otherwise. When the invocation pauses for approval or Ask User, the SDK emits signed, versioned suspension evidence. The child Conversation is the canonical one-shot suspension authority; the generation job carries only a versioned projection for UI, rolling-deploy routing, and the existing resume endpoint. Current Event Actor hosts select generation protocol v2 automatically; a trusted pre-capability producer remains on history only for the mixed-version drain. A resume shares one identity between its Conversation claim and provider-owner CAS, clears the predecessor projection, and publishes a successor only after a re-pause is canonical. A pending interrupt takes precedence over expected-action evidence from the same segment; if that segment already applied the expected action, publishing its successor pause cold-marks the prior head until a later applied commit replaces it. An ambiguous projection write is accepted only after reading back the exact generation, action, and suspension. The provider-start CAS is written only after client reconstruction and immediately before the continuation gate opens, then retains its exact execution identity after drain, so terminal recovery can compensate a projected claim only when that identity proves execution never began. Durable approval projection is exposed before the persistence barrier opens, preventing a resolved action from being announced afterward. Terminal no-action retirement cancels or settles the exact suspension and releases its delivery-side action admission before public settlement; if retention already removed the child Conversation, the delivery remains authoritative for its exact admission identity. Resume, re-pause, cancellation, and expiry claim or replace that exact suspension before touching its job projection, so later mailbox deliveries stay blocked until terminal history and handling evidence settle.

  • Event actor receipt: the private, terminal proof stored on the authoritative AgentTriggerDelivery row for one bound actor invocation. Its unique delivery identity, terminal resolution, exact checkpoint, and bounded action identity provide replay and recovery for the retention window without storing prompts, events, tool arguments, tool output, or conversation history. It does not own the actor checkpoint; the conversation keeps only the actor head and any active unresolved reconciliation until this receipt is durable.

  • Agent event actor mailbox: the automatic durable delivery-ordering lane for one authenticated source binding. It keeps later deliveries queued after transport admission until the current child turn records an authoritative terminal handling outcome. It serializes existing coalesced batches and individual events without becoming a second execution controller or actor checkpoint store.

  • Agent trigger capability shield: the durable mixed-version representation for internal trigger work that only a capability-aware worker may execute. Mongo uses an old-publishable staging shell; a queued leased shell without an owner or deadline, which old workers cannot claim but can use for bounded lane rechecks; a private lease only during execution; and a legacy-terminal capability_dead shell once dead. Private capability fields own current claiming, retry, and dead-letter truth. Redis uses a versioned fail-closed terminal status and recovery index that old replacement scripts and sweepers cannot consume. The shield is an implementation detail at the storage seam, never a deployment switch or user-configured product mode.

  • Theme definition: a versioned, data-only description of LibreChat semantic colors and shared appearance roles, optionally specialized by light or dark mode. The theme module validates and resolves partial definitions against bundled defaults before adapters apply them. A theme definition does not contain arbitrary CSS, application behavior, or alternate feature layouts.