🪗 feat: Compact Context in a Summarize-Only Turn (#15690)

* 🗜️ feat: Manual Context Compaction as a Summarize-Only Turn

Adds a user-triggered "Compact context" action that summarizes the active
branch on demand. The compaction is an ordinary agent turn: the client
submits it through the normal chat pipeline with `compact: true`, the
controller hangs it off the branch's leaf instead of creating a user
message, and the agent client runs the graph summarize-only (the SDK's new
`summarizeOnly` input), so the summary streams into the response
placeholder under the leaf exactly like the automatic detour's and is
persisted as the boundary every later turn starts from.

Nothing new is needed for job registration, billing, abort, persistence,
or the usage snapshot: the run emits the same summary step and post-summary
context snapshot the automatic path already emits, and the response message
carries the summary part plus the `summaryUsedTokens` baseline the gauge
reads. The controller validates the request (existing conversation, a real
leaf, summarization enabled, no edit/regenerate/continue combination) and
answers 409 when the leaf is already a bare summary.

Client-side the submission borrows the regenerate shape — no new user
bubble, the placeholder parents onto the leaf — while the wire payload
drops `isRegenerate` so the server never treats it as a regenerated user
turn. The action sits in the context-usage popover, gated by the same
`summarization.enabled` switch as the automatic detour.

* 🗜️ fix: Localize a Compaction That Cannot Run

A compaction the graph could not attempt (summarization disabled, the
instructions already exceeding the budget, or an explicit recency window
covering the whole conversation) rejects with the SDK's
`ManualSummarizationSkippedError`, which rendered as the generic admin
error. It now maps to a typed `compaction_skipped` payload with the reason,
and a run that ends with neither a summary nor a recorded error fails as
`compaction_failed`; the Error component localizes both. A run that already
recorded why it stopped keeps that error part instead of failing twice.

* 🗜️ fix: Anchor a Compaction on the Leaf Everywhere It Is Read

Review round 1.

- The client sent the leaf's parent as `parentMessageId`, which the
  controller treats as the anchor, so the summary would have skipped the
  latest reply and landed as its sibling. The leaf's own id is now both the
  placeholder's parent and the server-side anchor.
- A compaction has no user message of its own, and every consumer of the
  job's user-message slot now knows it: the error path parents its row on
  the loaded anchor and never upserts the leaf, and the job records the
  regenerate shape so a reconnecting client rebuilds the turn under the
  leaf instead of recreating it as a user message.
- The controller no longer reads the anchor before the branch is loaded;
  `BaseClient` validates it against the history it loads anyway and
  answers 404 for a missing leaf or a typed `nothing_to_summarize` for a
  finished compaction. That predicate is shared through the data provider
  and ignores a summary still streaming or one that failed, so an
  interrupted compaction can be retried.
- A compaction never consumes files staged in the composer, the action is
  shown only when the server advertises `compactionEnabled`, and the
  in-flight marker is feature-local Jotai state rather than a read of the
  Recoil submission store.

* 🗜️ fix: Keep a Compaction Turn to Its Summary

Review round 2, plus the class it belongs to. A compaction submits no user
turn, so nothing that keys off one may run: the memory pass no longer runs
over the history (it would have spent a model call and could repeat memory
writes), and skill primes are no longer injected into the transcript the
run is about to summarize.

The client's in-flight marker is also reconciled on mount: a compaction
that finished while the view was away left the marker set, and the next
ordinary turn in that conversation would have shown as compacting.

* 🧹 chore: Sort Imports in the Compaction Hook Spec

* 🔧 chore: Update @librechat/agents dependency to version 3.8.1 in package-lock.json and package.json files
This commit is contained in:
Danny Avila 2026-09-07 21:51:54 -04:00 • committed by GitHub
parent f1bd85498e
commit 3e2a6f333b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
30 changed files with 1059 additions and 77 deletions

View file

@ -24,7 +24,9 @@ const {
Constants,
FileSources,
Tools,
ErrorTypes,
ContentTypes,
isCompactedLeaf,
excludedKeys,
EModelEndpoint,
mergeFileConfig,
@ -438,6 +440,10 @@ class BaseClient {
const [overrideConvoId, overrideUserMessageId] = this.processOverideIds();
const { isEdited, isContinued } = opts;
if (opts.isCompaction === true) {
/** The leaf stands in for the user message and is already persisted. */
this.skipSaveUserMessage = true;
}
const user = opts.user ?? null;
this.user = user;
const saveOptions = this.getSaveOptions();
@ -497,6 +503,64 @@ class BaseClient {
};
}
/**
* The message a compaction turn hangs off: the branch's leaf, presented in
* the user-message slot so the response parents onto it and every consumer
* of `userMessage` (progress, job metadata, the abort path) keeps working.
* Identity fields only: the row stays in history untouched, and the object
* mirrored into job metadata must not carry the leaf's full content.
* @param {string} parentMessageId
* @returns {TMessage}
*/
getCompactionAnchor(parentMessageId) {
const leaf = this.currentMessages[this.currentMessages.length - 1];
if (leaf == null || leaf.messageId !== parentMessageId) {
throw Object.assign(new Error('The message to compact up to was not found.'), {
statusCode: 404,
code: 'COMPACTION_ANCHOR_NOT_FOUND',
});
}
if (isCompactedLeaf(leaf)) {
/** Typed so a stream that already started renders localized copy. */
throw Object.assign(
new Error(
JSON.stringify({
type: ErrorTypes.COMPACTION_SKIPPED,
reason: 'nothing_to_summarize',
}),
),
{ statusCode: 409, code: 'NOTHING_TO_COMPACT' },
);
}
return {
messageId: leaf.messageId,
parentMessageId: leaf.parentMessageId,
conversationId: leaf.conversationId,
isCreatedByUser: leaf.isCreatedByUser === true,
text: '',
};
}
/**
* The message the turn hangs off: a fresh user message, the edited message
* already in history, or (for a compaction) the branch's leaf.
* @returns {TMessage}
*/
resolveStartUserMessage({ opts, message, userMessageId, parentMessageId, conversationId }) {
if (opts.isCompaction) {
return this.getCompactionAnchor(parentMessageId);
}
if (opts.isEdited) {
return this.currentMessages[this.currentMessages.length - 2];
}
return this.createUserMessage({
messageId: userMessageId,
parentMessageId,
conversationId,
text: message,
});
}
async handleStartMethods(message, opts) {
const {
user,
@ -510,14 +574,13 @@ class BaseClient {
} = await this.setMessageOptions(opts);
this.options.startupTelemetry?.mark('history_loaded');
const userMessage = opts.isEdited
? this.currentMessages[this.currentMessages.length - 2]
: this.createUserMessage({
messageId: userMessageId,
parentMessageId,
conversationId,
text: message,
});
const userMessage = this.resolveStartUserMessage({
opts,
message,
userMessageId,
parentMessageId,
conversationId,
});
/**
* Attach quoted excerpts (the "Add to chat" selections from `req.body.quotes`)
@ -527,7 +590,7 @@ class BaseClient {
* merged into the model-facing text later, per message, in `buildMessages`,
* keeping the stored `text` clean while the count stays consistent.
*/
if (!opts.isEdited) {
if (!opts.isEdited && !opts.isCompaction) {
const referencedQuotes = getReferencedQuotes(this.options.req?.body?.quotes);
if (referencedQuotes != null) {
userMessage.quotes = referencedQuotes;
@ -729,7 +792,7 @@ class BaseClient {
}
}
this.continued = true;
} else {
} else if (opts.isCompaction !== true) {
this.currentMessages.push(userMessage);
}
@ -771,7 +834,8 @@ class BaseClient {
this.assertBuiltModelBoundContent(payload);
this.options.startupTelemetry?.mark('messages_built');
if (tokenCountMap && tokenCountMap[userMessage.messageId]) {
/** A compaction anchor is the persisted leaf, whose own count must stay. */
if (tokenCountMap && tokenCountMap[userMessage.messageId] && opts.isCompaction !== true) {
userMessage.tokenCount = tokenCountMap[userMessage.messageId];
logger.debug('[BaseClient] userMessage', {
messageId: userMessage.messageId,
@ -1092,6 +1156,7 @@ class BaseClient {
}
if (
opts.isCompaction !== true &&
this.contextMeta?.calibrationRatio > 0 &&
this.contextMeta.calibrationRatio !== 1 &&
userMessage.tokenCount > 0

View file

@ -3084,3 +3084,140 @@ describe('BaseClient', () => {
});
});
});
describe('BaseClient compaction turns', () => {
const compactionOptions = { modelOptions: { model: 'gpt-4o-mini', temperature: 0 } };
const compactionHistory = [
{ role: 'user', isCreatedByUser: true, text: 'Hello', messageId: 'u1' },
{
role: 'assistant',
isCreatedByUser: false,
text: 'Hi',
messageId: 'a1',
parentMessageId: 'u1',
tokenCount: 7,
},
];
let CompactClient;
beforeEach(() => {
jest.clearAllMocks();
CompactClient = initializeFakeClient(apiKey, compactionOptions, compactionHistory);
});
test('presents the leaf as the user message and never re-saves it', async () => {
const result = await CompactClient.handleStartMethods('', {
conversationId: 'convo-compact',
parentMessageId: 'a1',
preallocatedUserMessageId: 'a1',
isCompaction: true,
});
expect(result.userMessage).toEqual({
messageId: 'a1',
parentMessageId: 'u1',
conversationId: undefined,
isCreatedByUser: false,
text: '',
});
expect(CompactClient.skipSaveUserMessage).toBe(true);
/** History is loaded through the leaf, and nothing is appended to it. */
expect(CompactClient.currentMessages.map((message) => message.messageId)).toEqual(['u1', 'a1']);
});
test('refuses a compaction whose anchor is not the loaded leaf', async () => {
await expect(
CompactClient.handleStartMethods('', {
conversationId: 'convo-compact',
parentMessageId: 'missing',
preallocatedUserMessageId: 'missing',
isCompaction: true,
}),
).rejects.toMatchObject({ statusCode: 404, code: 'COMPACTION_ANCHOR_NOT_FOUND' });
});
test('refuses to compact a branch whose leaf is already a finished compaction', async () => {
const compacted = [
...compactionHistory,
{
role: 'assistant',
isCreatedByUser: false,
text: '',
messageId: 's1',
parentMessageId: 'a1',
content: [
{
type: ContentTypes.SUMMARY,
content: [{ type: ContentTypes.TEXT, text: 'checkpoint' }],
},
],
},
];
CompactClient = initializeFakeClient(apiKey, compactionOptions, compacted);
await expect(
CompactClient.handleStartMethods('', {
conversationId: 'convo-compact',
parentMessageId: 's1',
preallocatedUserMessageId: 's1',
isCompaction: true,
}),
).rejects.toMatchObject({
statusCode: 409,
code: 'NOTHING_TO_COMPACT',
message: JSON.stringify({ type: 'compaction_skipped', reason: 'nothing_to_summarize' }),
});
});
test('lets an interrupted compaction be retried', async () => {
const interrupted = [
...compactionHistory,
{
role: 'assistant',
isCreatedByUser: false,
text: '',
messageId: 's1',
parentMessageId: 'a1',
content: [{ type: ContentTypes.SUMMARY, content: [], summarizing: true }],
},
];
CompactClient = initializeFakeClient(apiKey, compactionOptions, interrupted);
const result = await CompactClient.handleStartMethods('', {
conversationId: 'convo-compact',
parentMessageId: 's1',
preallocatedUserMessageId: 's1',
isCompaction: true,
});
expect(result.userMessage.messageId).toBe('s1');
});
test('parents the response onto the leaf and persists only the response', async () => {
const saveSpy = jest.spyOn(CompactClient, 'saveMessageToDatabase').mockResolvedValue({});
const updateSpy = jest.spyOn(CompactClient, 'updateMessageInDatabase').mockResolvedValue({});
/** A calibration ratio and a counted anchor would, on an ordinary turn,
* rewrite the user message's persisted count; the leaf's must survive. */
CompactClient.contextMeta = { calibrationRatio: 0.5 };
CompactClient.buildMessages = jest.fn(async () => ({
prompt: [],
tokenCountMap: { a1: 7 },
promptTokens: 7,
}));
const response = await CompactClient.sendMessage('', {
conversationId: 'convo-compact',
parentMessageId: 'a1',
preallocatedUserMessageId: 'a1',
isCompaction: true,
});
expect(response.parentMessageId).toBe('a1');
expect(response.isCreatedByUser).toBe(false);
expect(saveSpy).toHaveBeenCalledTimes(1);
expect(saveSpy.mock.calls[0][0].messageId).toBe(response.messageId);
expect(updateSpy).not.toHaveBeenCalled();
expect(CompactClient.currentMessages.map((message) => message.messageId)).toEqual(['u1', 'a1']);
expect(compactionHistory[1].tokenCount).toBe(7);
});
});

View file

@ -46,7 +46,7 @@
"@azure/storage-blob": "^12.30.0",
"@google/genai": "^2.8.0",
"@keyv/redis": "5.1.6",
"@librechat/agents": "^3.8.0",
"@librechat/agents": "^3.8.1",
"@librechat/api": "*",
"@librechat/data-schemas": "*",
"@microsoft/microsoft-graph-client": "^3.0.7",

View file

@ -176,6 +176,7 @@ const {
UsageEvents,
Permissions,
VisionModes,
ErrorTypes,
ContentTypes,
FileSources,
ApprovalEvents,
@ -318,19 +319,48 @@ function captureRunContextMeta(client) {
});
}
/** Text of a summary content part; empty for anything else. */
function getSummaryPartText(part) {
if (part?.type !== ContentTypes.SUMMARY || !Array.isArray(part.content)) {
return '';
}
return part.content
.map((block) => (typeof block?.text === 'string' ? block.text : ''))
.join('')
.trim();
}
/**
* A compaction turn's response is its summary. The run emits no text, so a
* completion without a usable summary part means the summarizer produced
* nothing. A run that already recorded why (an error part, e.g. a skipped
* compaction) persists with that explanation; one that ended with neither
* fails as a typed error instead of persisting an empty assistant message.
* @param {Array<import('librechat-data-provider').TMessageContentParts>} contentParts
*/
function markCompactionSummary(contentParts) {
const summary = contentParts.find(
(part) => part?.failed !== true && getSummaryPartText(part).length > 0,
);
if (summary != null) {
summary.initiatedBy = 'user';
return;
}
if (contentParts.some((part) => part?.type === ContentTypes.ERROR)) {
return;
}
throw Object.assign(new Error(JSON.stringify({ type: ErrorTypes.COMPACTION_FAILED })), {
code: 'COMPACTION_FAILED',
});
}
function getLatestEventActorSummary(contentParts) {
if (!Array.isArray(contentParts)) {
return undefined;
}
for (let index = contentParts.length - 1; index >= 0; index -= 1) {
const part = contentParts[index];
if (part?.type !== ContentTypes.SUMMARY || !Array.isArray(part.content)) {
continue;
}
const text = part.content
.map((block) => (typeof block?.text === 'string' ? block.text : ''))
.join('')
.trim();
const text = getSummaryPartText(part);
if (text.length === 0) {
continue;
}
@ -349,6 +379,10 @@ function getLatestEventActorSummary(contentParts) {
* still reaches the logs through `getSafeErrorMetadata`.
*/
function getUserFacingRequestError(baseMessage, error, appConfig) {
/** Carries no model or user content, so it is safe under every filter. */
if (error?.name === 'ManualSummarizationSkippedError') {
return JSON.stringify({ type: ErrorTypes.COMPACTION_SKIPPED, reason: error.reason });
}
const protectionEnabled = hasModelBoundContentProtection(
appConfig?.filters,
appConfig?.messageFilter?.pii,
@ -3502,10 +3536,18 @@ class AgentClient extends BaseClient {
});
const completion = filterMalformedContentParts(this.contentParts);
if (this.isCompactionTurn()) {
markCompactionSummary(completion);
}
const metadata = this.buildResponseMetadata();
return metadata ? { completion, metadata } : { completion };
}
/** A manual compaction runs the graph summarize-only: the summary is the response. */
isCompactionTurn() {
return this.options?.req?.body?.compact === true;
}
/**
* Assembles the response message `metadata`: Vertex thought signatures plus
* the persisted context breakdown (Part A) and the usage/cost rollup (Part B),
@ -4466,8 +4508,16 @@ class AgentClient extends BaseClient {
* synthetic prefix. Names NOT primed this turn still reconstruct from
* history, preserving sticky manual re-priming across turns.
*/
const manualSkillPrimes = this.options.agent?.manualSkillPrimes;
const alwaysApplySkillPrimes = this.options.agent?.alwaysApplySkillPrimes;
/** A compaction summarizes what was already said. No user turn was
* submitted, so it primes no skills into the transcript it is about
* to summarize and runs no memory pass over it. */
const isCompactionTurn = this.isCompactionTurn();
const manualSkillPrimes = isCompactionTurn
? undefined
: this.options.agent?.manualSkillPrimes;
const alwaysApplySkillPrimes = isCompactionTurn
? undefined
: this.options.agent?.alwaysApplySkillPrimes;
const freshSkillPrimeNames = collectFreshSkillPrimeNames({
manualSkillPrimes,
alwaysApplySkillPrimes,
@ -4602,7 +4652,7 @@ class AgentClient extends BaseClient {
});
const memoryMessages =
this.processMemory && this.memoryPayload
this.processMemory && this.memoryPayload && !isCompactionTurn
? formatAgentMessages(
stripActivityLabelParts(this.memoryPayload),
undefined,
@ -4661,7 +4711,7 @@ class AgentClient extends BaseClient {
// messages = addCacheControl(messages);
// }
if (this.processMemory) {
if (this.processMemory && !isCompactionTurn) {
memoryPromise = this.runMemory(memoryMessages);
}
@ -4787,6 +4837,7 @@ class AgentClient extends BaseClient {
traceContext: buildTraceContext(this.options),
tenantId: resolveRequestTenantId(this.options.req ?? {}),
summarizationConfig: appConfig?.summarization,
summarizeOnly: this.isCompactionTurn(),
appConfig,
tokenCounter,
/** Bills subagent child-run model calls — foreground usage joins

View file

@ -149,6 +149,81 @@ function getPreliminaryResponseMessageId({ messageId, responseMessageId }) {
return `${messageId.replace(/_+$/, '')}_`;
}
/**
* Manual compaction runs as a summarize-only turn hung off the branch's leaf.
* It needs an existing branch to summarize, and it cannot be combined with the
* turn shapes that create or rewrite a user message.
* @returns {{ status: number, code: string, error: string } | null}
*/
function getCompactionRejection(req, { conversationId, parentMessageId }) {
if (req.config?.summarization?.enabled === false) {
return {
status: 400,
code: 'COMPACTION_DISABLED',
error: 'Context compaction is disabled for this deployment.',
};
}
if (!conversationId || conversationId === Constants.NEW_CONVO) {
return {
status: 400,
code: 'INVALID_COMPACTION_REQUEST',
error: 'Compaction requires an existing conversation.',
};
}
if (
typeof parentMessageId !== 'string' ||
parentMessageId.length === 0 ||
parentMessageId === Constants.NO_PARENT
) {
return {
status: 400,
code: 'INVALID_COMPACTION_REQUEST',
error: 'Compaction requires the message to compact up to.',
};
}
const { isContinued, isRegenerate, editedContent, responseMessageId } = req.body ?? {};
if (isContinued || isRegenerate || editedContent != null || responseMessageId) {
return {
status: 400,
code: 'INVALID_COMPACTION_REQUEST',
error: 'Compaction cannot be combined with an edit, regenerate, or continue.',
};
}
return null;
}
/**
* The leaf a compaction hangs off, in the user-message slot the job metadata
* and the abort path read before the branch is loaded. Identity only: the
* client validates the leaf against the history it loads anyway, and the job
* must not carry the leaf's content.
*/
function projectCompactionAnchor({ messageId, conversationId }) {
return { messageId, conversationId, text: '' };
}
/**
* The id the turn's user message is created under. A compaction creates no
* user message: its "user message" slot holds the leaf it summarizes up to,
* and a bound event turn keys the id off its task.
* @returns {string}
*/
function resolvePreallocatedUserMessageId({
isCompaction,
parentMessageId,
eventTaskId,
overrideUserMessageId,
overrideParentMessageId,
}) {
if (isCompaction) {
return parentMessageId;
}
if (eventTaskId != null) {
return `${eventTaskId}:user`;
}
return overrideUserMessageId ?? overrideParentMessageId ?? crypto.randomUUID();
}
function getPreliminaryUserMessage(
{ messageId, parentMessageId, text, quotes, files, manualSkills, alwaysAppliedSkills },
conversationId,
@ -292,7 +367,16 @@ async function saveErrorTurn(
let userMessage = null;
let errorMessageId = null;
let errorParentMessageId = null;
if (isRegenerate) {
if (req.body?.compact === true) {
/** The anchor is the persisted leaf, never rewritten. Without the
* loaded anchor (the branch failed to load) there is nothing safe
* to parent an error row onto, so nothing is written. */
if (liveUserMessage?.messageId == null) {
return;
}
errorMessageId = getPreliminaryResponseMessageId({ messageId: liveUserMessage.messageId });
errorParentMessageId = liveUserMessage.messageId;
} else if (isRegenerate) {
errorMessageId =
typeof responseMessageId === 'string' && responseMessageId.length > 0
? responseMessageId
@ -615,6 +699,22 @@ const ResumableAgentController = async (req, res, next, initializeClient, addTit
const userId = req.user.id;
const tenantId = req.user.tenantId;
const isCompaction = req.body?.compact === true;
if (isCompaction) {
const rejection = getCompactionRejection(req, {
conversationId: reqConversationId,
parentMessageId,
});
if (rejection) {
startupTelemetry?.end('rejected');
return sendGenerationJson(
res,
rejection.status,
{ code: rejection.code, error: rejection.error },
generationProtocolVersion,
);
}
}
const rawClientRequestId = req.body?.clientRequestId;
if (
rawClientRequestId != null &&
@ -1355,10 +1455,13 @@ const ResumableAgentController = async (req, res, next, initializeClient, addTit
if (eventTaskId != null) {
req._agentEventTaskId = eventTaskId;
}
const preallocatedUserMessageId =
eventTaskId == null
? (overrideUserMessageId ?? overrideParentMessageId ?? crypto.randomUUID())
: `${eventTaskId}:user`;
const preallocatedUserMessageId = resolvePreallocatedUserMessageId({
isCompaction,
parentMessageId,
eventTaskId,
overrideUserMessageId,
overrideParentMessageId,
});
const overrideConversationId = rawOverrideConversationId
? rawOverrideConversationId.split(Constants.COMMON_DIVIDER)[0]
: undefined;
@ -1441,11 +1544,13 @@ const ResumableAgentController = async (req, res, next, initializeClient, addTit
const endpointIconURL = getEndpointIconURL(req, endpointOption);
const responseModel = getAgentResponseModel(req, endpointOption);
const preliminaryUserMessage = getPreliminaryUserMessage(
{ ...req.body, messageId: preallocatedUserMessageId },
conversationId,
req._agentEventTriggerProjection,
);
const preliminaryUserMessage = isCompaction
? projectCompactionAnchor({ messageId: parentMessageId, conversationId })
: getPreliminaryUserMessage(
{ ...req.body, messageId: preallocatedUserMessageId },
conversationId,
req._agentEventTriggerProjection,
);
const job = await GenerationJobManager.createJob(streamId, userId, conversationId, {
startupTelemetry,
...(recoveredSteerId && { recoveredSteerId }),
@ -1493,7 +1598,10 @@ const ResumableAgentController = async (req, res, next, initializeClient, addTit
}),
}),
}),
...(isRegenerate && { isRegenerate: true }),
/** A compaction is regenerate-shaped for every consumer of the job:
* no user message of its own, the response parented onto an
* existing message. A reconnecting client rebuilds it that way. */
...((isRegenerate || isCompaction) && { isRegenerate: true }),
...(scheduleId
? {
scheduleId,
@ -2149,6 +2257,7 @@ const ResumableAgentController = async (req, res, next, initializeClient, addTit
getReqData,
isContinued,
isRegenerate,
isCompaction,
editedContent,
conversationId,
parentMessageId,

View file

@ -308,6 +308,7 @@ router.get('/', async function (req, res) {
langfuseFanoutEnabled,
langfuseConnectionAccess,
insightsEnabled: isEnabled(process.env.ENABLE_INSIGHTS),
compactionEnabled: appConfig?.summarization?.enabled !== false,
...(cloudFront ? { cloudFront } : {}),
...(rum ? { rum } : {}),
fileUploadSseEnabled: isEnabled(process.env.FILE_UPLOAD_SSE_ENABLED),

View file

@ -334,6 +334,13 @@ export type TOptions = {
isRegenerate?: boolean;
isContinued?: boolean;
isEdited?: boolean;
/**
* Manual context compaction: a summarize-only turn hung off the branch's
* leaf (`messageId`). Shaped like a regenerate on the client — no new user
* bubble, the response placeholder parents onto the leaf — and sent with
* `compact: true` so the server runs the graph summarize-only.
*/
compact?: boolean;
overrideMessages?: t.TMessage[];
/**
* Authoritative attachment list for this submission: a rerun replays the edited

View file

@ -0,0 +1,57 @@
import { memo, useId } from 'react';
import { ScrollText } from 'lucide-react';
import { Button, Spinner, TooltipAnchor } from '@librechat/client';
import { useLocalize } from '~/hooks';
interface CompactActionProps {
compact: () => void;
canCompact: boolean;
isCompacting: boolean;
}
/**
* Manual context compaction, surfaced where the context usage is already read.
* Presentational on purpose: the popover unmounts on hide, so the hook that
* drives the submission lives in the always-mounted indicator above.
*/
function CompactAction({ compact, canCompact, isCompacting }: CompactActionProps) {
const localize = useLocalize();
const descriptionId = useId();
const description = localize('com_ui_context_compact_info');
return (
<>
<TooltipAnchor
side="bottom"
description={description}
render={
<Button
type="button"
variant="outline"
onClick={compact}
disabled={!canCompact}
aria-busy={isCompacting}
aria-describedby={descriptionId}
className="h-8 w-full justify-center gap-2 text-sm"
>
{isCompacting ? (
<Spinner className="size-4" />
) : (
<ScrollText className="size-4" aria-hidden="true" />
)}
{isCompacting
? localize('com_ui_context_compacting')
: localize('com_ui_context_compact')}
</Button>
}
/>
{/* Ariakit's tooltip anchor sets no `aria-describedby`, and the tooltip
exists only while hovered, so the copy stays reachable here. */}
<span id={descriptionId} className="sr-only">
{description}
</span>
</>
);
}
export default memo(CompactAction);

View file

@ -1,10 +1,13 @@
import { memo, useCallback, useEffect, useRef, useState } from 'react';
import * as Ariakit from '@ariakit/react';
import { Spinner } from '@librechat/client';
import { Constants } from 'librechat-data-provider';
import type { TConversation } from 'librechat-data-provider';
import type { CurrencyConfig } from '~/utils';
import useCompactConversation, { supportsCompaction } from '~/hooks/Chat/useCompactConversation';
import { useGetLangfuseSessionLinkQuery, useGetStartupConfig } from '~/data-provider';
import useTokenUsage from '~/hooks/Chat/useTokenUsage';
import CompactAction from './CompactAction';
import { formatTokens, cn } from '~/utils';
import { useLocalize } from '~/hooks';
import Breakdown from './Breakdown';
@ -28,13 +31,18 @@ function TokenUsageIndicator({
showCost,
currency,
langfuseConnectionAccess,
compactionEnabled,
}: TokenUsageProps & {
showCost: boolean;
currency?: CurrencyConfig;
langfuseConnectionAccess: boolean;
compactionEnabled: boolean;
}) {
const localize = useLocalize();
const view = useTokenUsage({ index, conversation, isSubmitting });
/** Owned here, not in the popover: `unmountOnHide` would otherwise lose the
* in-flight state the moment the pointer leaves. */
const compaction = useCompactConversation();
const popover = Ariakit.usePopoverStore({ placement: 'top' });
const popoverOpen = Ariakit.useStoreState(popover, 'open');
const disclosureRef = useRef<HTMLButtonElement>(null);
@ -124,13 +132,17 @@ function TokenUsageIndicator({
}
const hasMax = view.maxTokens != null && view.maxTokens > 0;
const ariaLabel = hasMax
const usageAriaLabel = hasMax
? localize('com_ui_context_usage_label', {
0: formatTokens(view.usedTokens),
1: formatTokens(view.maxTokens ?? 0),
2: String(Math.round(view.percent)),
})
: localize('com_ui_context_usage_label_unknown', { 0: formatTokens(view.usedTokens) });
const ariaLabel = compaction.isCompacting
? localize('com_ui_context_compacting')
: usageAriaLabel;
const showCompactingIndicator = compaction.isCompacting && !popoverOpen;
return (
<>
@ -144,6 +156,7 @@ function TokenUsageIndicator({
type="button"
data-testid="token-usage"
aria-label={ariaLabel}
aria-busy={compaction.isCompacting}
aria-haspopup="dialog"
onPointerDown={() => {
pinAtPointerDownRef.current = pinnedRef.current;
@ -183,16 +196,20 @@ function TokenUsageIndicator({
'duration-300 animate-in fade-in zoom-in-95',
)}
>
<span
role="meter"
aria-valuemin={0}
aria-valuemax={hasMax ? view.maxTokens : undefined}
aria-valuenow={view.usedTokens}
aria-label={localize('com_ui_context_usage')}
className="flex items-center justify-center"
>
<Gauge percent={view.percent} indeterminate={!hasMax} />
</span>
{showCompactingIndicator ? (
<Spinner className="size-5 text-text-secondary" />
) : (
<span
role="meter"
aria-valuemin={0}
aria-valuemax={hasMax ? view.maxTokens : undefined}
aria-valuenow={view.usedTokens}
aria-label={localize('com_ui_context_usage')}
className="flex items-center justify-center"
>
<Gauge percent={view.percent} indeterminate={!hasMax} />
</span>
)}
</Ariakit.PopoverDisclosure>
{/* Focus the labelled dialog on keyboard/click open so screen readers
enter and announce the breakdown, and so focus stays contained instead
@ -220,12 +237,26 @@ function TokenUsageIndicator({
'data-[leave]:translate-y-1 data-[leave]:scale-95 data-[leave]:opacity-0',
)}
>
<Breakdown
view={view}
showCost={showCost}
currency={currency}
langfuseSessionUrl={langfuseSession?.url ?? undefined}
/>
{/* The popover owns its width, which the breakdown held only while it
was the sole child of a shrink-to-fit box. */}
<div className="w-72 space-y-3">
<Breakdown
view={view}
showCost={showCost}
currency={currency}
langfuseSessionUrl={langfuseSession?.url ?? undefined}
/>
{compactionEnabled && supportsCompaction(conversation?.endpoint) && (
<>
<div className="border-t border-border-light" role="separator" />
<CompactAction
compact={compaction.compact}
canCompact={compaction.canCompact}
isCompacting={compaction.isCompacting}
/>
</>
)}
</div>
</Ariakit.Popover>
</>
);
@ -246,6 +277,11 @@ const TokenUsage = memo(function TokenUsage(props: TokenUsageProps) {
showCost={startupConfig.interface?.contextCost === true}
currency={startupConfig.interface?.currency}
langfuseConnectionAccess={startupConfig.langfuseConnectionAccess === true}
/** Same `summarization.enabled` switch that governs the automatic detour,
* advertised positively: a server that does not know the capability
* (a cached config from an older release) must not receive the request,
* which it would run as an empty, billed ordinary turn. */
compactionEnabled={startupConfig.compactionEnabled === true}
/>
);
});

View file

@ -165,6 +165,7 @@ const Part = memo(function Part({
model={part.model}
provider={part.provider}
tokenCount={part.tokenCount}
initiatedBy={part.initiatedBy}
summarizing={part.summarizing}
failed={part.failed}
/>

View file

@ -12,7 +12,7 @@ import { cn } from '~/utils';
type SummaryProps = Pick<
SummaryContentPart,
'content' | 'model' | 'provider' | 'tokenCount' | 'summarizing' | 'failed'
'content' | 'model' | 'provider' | 'tokenCount' | 'summarizing' | 'failed' | 'initiatedBy'
>;
function useCopyToClipboard(content?: string) {
@ -204,7 +204,7 @@ const FloatingSummaryBar = memo(
);
const Summary = memo(
({ content, model, provider, tokenCount, summarizing, failed }: SummaryProps) => {
({ content, model, provider, tokenCount, summarizing, failed, initiatedBy }: SummaryProps) => {
const contentId = useId();
const localize = useLocalize();
const [isExpanded, setIsExpanded] = useState(false);
@ -259,10 +259,13 @@ const Summary = memo(
if (isActivelyStreaming) {
return localize('com_ui_summarizing');
}
return failed
? localize('com_ui_summarize_failed')
if (failed) {
return localize('com_ui_summarize_failed');
}
return initiatedBy === 'user'
? localize('com_ui_context_compacted_by_you')
: localize('com_ui_conversation_summarized');
}, [isActivelyStreaming, failed, localize]);
}, [isActivelyStreaming, failed, initiatedBy, localize]);
if (!summarizing && !text) {
return null;

View file

@ -65,6 +65,17 @@ type TContextOverflow = {
availableMessageTokens?: number;
};
type TCompactionSkipped = {
reason?: string;
};
/** Why a manual compaction could not run, keyed by the SDK's skip reason. */
const compactionSkippedKeys: Record<string, TranslationKeys> = {
disabled: 'com_error_compaction_disabled',
instructions_exceed_budget: 'com_error_compaction_budget',
nothing_to_summarize: 'com_error_compaction_nothing',
};
/**
* SDK boilerplate already covered by the localized headline; whatever remains
* (specific guidance and the token budget breakdown) renders as details.
@ -118,6 +129,9 @@ const errorMessages = {
[ErrorTypes.STREAM_EXPIRED]: 'com_error_stream_expired',
[ErrorTypes.MODEL_NOT_FOUND]: langChainErrorKeys.MODEL_NOT_FOUND,
[ErrorTypes.MODEL_RATE_LIMIT]: langChainErrorKeys.MODEL_RATE_LIMIT,
[ErrorTypes.COMPACTION_FAILED]: 'com_error_compaction_failed',
[ErrorTypes.COMPACTION_SKIPPED]: (json: TCompactionSkipped, localize: LocalizeFunction) =>
localize(compactionSkippedKeys[json.reason ?? ''] ?? 'com_error_compaction_failed'),
[ErrorTypes.EMPTY_MESSAGES]: (json: TGenericError, localize: LocalizeFunction) => {
const detail = emptyMessagesBoilerplate
.reduce((info, sentence) => info.replace(sentence, ''), json.info ?? '')

View file

@ -130,3 +130,29 @@ describe('Error: agent context budget errors', () => {
expect(screen.getByText(catalog.com_error_final_context_overflow)).toBeInTheDocument();
});
});
describe('Error — manual compaction', () => {
it('renders the failed-compaction copy', () => {
render(<Error text={JSON.stringify({ type: ErrorTypes.COMPACTION_FAILED })} />);
expect(screen.getByText(catalog.com_error_compaction_failed)).toBeInTheDocument();
});
it.each([
['disabled', 'com_error_compaction_disabled'],
['instructions_exceed_budget', 'com_error_compaction_budget'],
['nothing_to_summarize', 'com_error_compaction_nothing'],
])('renders the copy for a compaction skipped because %s', (reason, key) => {
render(<Error text={JSON.stringify({ type: ErrorTypes.COMPACTION_SKIPPED, reason })} />);
expect(screen.getByText(catalog[key])).toBeInTheDocument();
});
it('falls back to the failed copy for an unknown skip reason', () => {
render(
<Error text={JSON.stringify({ type: ErrorTypes.COMPACTION_SKIPPED, reason: 'exhausted' })} />,
);
expect(screen.getByText(catalog.com_error_compaction_failed)).toBeInTheDocument();
});
});

View file

@ -408,3 +408,105 @@ describe('useChatFunctions ask attachments', () => {
expect(isPasteSubmitted('queued-override-temp-file')).toBe(true);
});
});
describe('useChatFunctions ask compaction', () => {
beforeEach(() => {
jest.clearAllMocks();
mockGetQueryData.mockReturnValue({});
});
it('submits a summarize-only turn hung off the leaf without a new user bubble', () => {
const messages = [userMessage('u1'), assistantMessage('a1', 'u1')];
const { result, setMessages, setSubmission } = renderAsk(messages);
act(() => {
result.current.ask(
{ text: '', conversationId: 'conversation-1', messageId: 'a1', parentMessageId: 'u1' },
{ compact: true },
);
});
expect(setSubmission).toHaveBeenCalledTimes(1);
const submission = setSubmission.mock.calls[0][0] as TSubmission;
expect(submission.compact).toBe(true);
expect(submission.isRegenerate).toBe(true);
expect(submission.initialResponse?.parentMessageId).toBe('a1');
expect(submission.initialResponse?.messageId).toBe('a1_');
/** The leaf stands in for the user message so an error lands under it. */
expect(submission.userMessage.messageId).toBe('a1');
expect(submission.userMessage.overrideParentMessageId).toBeNull();
expect(submission.userMessage.files).toBeUndefined();
expect(submission.messages.map((message) => message.messageId)).toEqual(['u1', 'a1']);
expect(submission.regenerateMessages?.map((message) => message.messageId)).toEqual([
'u1',
'a1',
]);
const rendered = setMessages.mock.calls[0][0] as TMessage[];
expect(rendered.map((message) => message.messageId)).toEqual(['u1', 'a1', 'a1_']);
});
it('does not re-attach the leaf files a user-message leaf carries', () => {
const leaf = { ...userMessage('u2', 'a1'), files: [{ file_id: 'f1' }] } as TMessage;
const messages = [userMessage('u1'), assistantMessage('a1', 'u1'), leaf];
const { result, setSubmission } = renderAsk(messages);
act(() => {
result.current.ask(
{ text: '', conversationId: 'conversation-1', messageId: 'u2', parentMessageId: 'a1' },
{ compact: true },
);
});
const submission = setSubmission.mock.calls[0][0] as TSubmission;
expect(submission.userMessage.files).toBeUndefined();
expect(submission.initialResponse?.parentMessageId).toBe('u2');
});
});
describe('useChatFunctions ask compaction and the composer', () => {
it('leaves files staged in the composer untouched', () => {
const setMessages = jest.fn();
const setSubmission = jest.fn();
const setFiles = jest.fn();
const files = new Map([
[
'staged-file',
{
file_id: 'staged-file',
filepath: '/uploads/staged-file',
filename: 'next-message.pdf',
type: 'application/pdf',
},
],
]) as unknown as Parameters<typeof useChatFunctions>[0]['files'];
const messages = [userMessage('u1'), assistantMessage('a1', 'u1')];
const { result } = renderHook(() =>
useChatFunctions({
isSubmitting: false,
latestMessage: messages[1],
conversation: conversation('conversation-1'),
getMessages: () => messages,
setMessages,
setSubmission,
files,
setFiles,
}),
);
act(() => {
result.current.ask(
{ text: '', conversationId: 'conversation-1', messageId: 'a1', parentMessageId: 'a1' },
{ compact: true },
);
});
const submission = setSubmission.mock.calls.at(-1)?.[0] as TSubmission;
expect(submission.compact).toBe(true);
expect(submission.userMessage.files).toBeUndefined();
expect(submission.userMessage.parentMessageId).toBe('a1');
expect(setFiles).not.toHaveBeenCalled();
expect(isPasteSubmitted('staged-file')).toBe(false);
});
});

View file

@ -0,0 +1,140 @@
import { getDefaultStore } from 'jotai';
import { ContentTypes } from 'librechat-data-provider';
import { renderHook, act } from '@testing-library/react';
import type { TMessage } from 'librechat-data-provider';
import useCompactConversation, { compactingConversationAtom } from '../useCompactConversation';
const mockAsk = jest.fn();
let mockContext: {
index: number;
isSubmitting: boolean;
conversation: { conversationId: string; endpoint: string } | null;
};
let mockLatestMessage: TMessage | null;
jest.mock('~/Providers', () => ({
useChatContext: () => ({ ...mockContext, ask: mockAsk }),
}));
jest.mock('~/hooks/Messages/useLatestMessage', () => ({
useLatestMessage: () => mockLatestMessage,
}));
type ContentPart = NonNullable<TMessage['content']>[number];
const summaryPart = (overrides: Record<string, unknown> = {}): ContentPart =>
({
type: ContentTypes.SUMMARY,
content: [{ type: ContentTypes.TEXT, text: 'checkpoint' }],
...overrides,
}) as ContentPart;
const leaf = (overrides: Partial<TMessage> = {}): TMessage =>
({
messageId: 'a1',
parentMessageId: 'u1',
conversationId: 'convo-1',
isCreatedByUser: false,
text: 'answer',
...overrides,
}) as TMessage;
describe('useCompactConversation', () => {
beforeEach(() => {
mockAsk.mockClear();
mockContext = {
index: 0,
isSubmitting: false,
conversation: { conversationId: 'convo-1', endpoint: 'openAI' },
};
mockLatestMessage = leaf();
getDefaultStore().set(compactingConversationAtom, null);
});
it('drops a marker left by a compaction that finished while the view was away', () => {
getDefaultStore().set(compactingConversationAtom, 'convo-1');
const hook = renderHook(() => useCompactConversation());
expect(hook.result.current.isCompacting).toBe(false);
/** The next ordinary turn in the same conversation is not a compaction. */
mockContext.isSubmitting = true;
hook.rerender();
expect(hook.result.current.isCompacting).toBe(false);
});
it('keeps the marker when it mounts into a compaction still streaming', () => {
getDefaultStore().set(compactingConversationAtom, 'convo-1');
mockContext.isSubmitting = true;
const hook = renderHook(() => useCompactConversation());
expect(hook.result.current.isCompacting).toBe(true);
mockContext.isSubmitting = false;
hook.rerender();
expect(hook.result.current.isCompacting).toBe(false);
expect(getDefaultStore().get(compactingConversationAtom)).toBeNull();
});
it('submits a compaction anchored on the leaf itself', () => {
const { result } = renderHook(() => useCompactConversation());
expect(result.current.canCompact).toBe(true);
act(() => result.current.compact());
/** The server compacts up to `parentMessageId`, so it must be the leaf,
* not the leaf's own parent, or the latest reply is left out. */
expect(mockAsk).toHaveBeenCalledWith(
{ text: '', conversationId: 'convo-1', messageId: 'a1', parentMessageId: 'a1' },
{ compact: true },
);
});
it.each([
[
'a new conversation',
() => (mockContext.conversation = { conversationId: 'new', endpoint: 'openAI' }),
],
['a submission in flight', () => (mockContext.isSubmitting = true)],
['no leaf', () => (mockLatestMessage = null)],
[
'a leaf that is already a finished compaction',
() => (mockLatestMessage = leaf({ text: '', content: [summaryPart()] })),
],
])('cannot compact with %s', (_label, arrange) => {
arrange();
const { result } = renderHook(() => useCompactConversation());
expect(result.current.canCompact).toBe(false);
act(() => result.current.compact());
expect(mockAsk).not.toHaveBeenCalled();
});
it.each([
['still streaming', summaryPart({ summarizing: true })],
['failed', summaryPart({ failed: true })],
])('lets an interrupted compaction (summary %s) be retried', (_label, part) => {
mockLatestMessage = leaf({ text: '', content: [part] });
const { result } = renderHook(() => useCompactConversation());
expect(result.current.canCompact).toBe(true);
});
it('reports compacting only for the conversation it submitted, until the turn settles', () => {
const hook = renderHook(() => useCompactConversation());
act(() => hook.result.current.compact());
mockContext.isSubmitting = true;
hook.rerender();
expect(hook.result.current.isCompacting).toBe(true);
mockContext.conversation = { conversationId: 'convo-2', endpoint: 'openAI' };
hook.rerender();
expect(hook.result.current.isCompacting).toBe(false);
mockContext.conversation = { conversationId: 'convo-1', endpoint: 'openAI' };
mockContext.isSubmitting = false;
hook.rerender();
expect(hook.result.current.isCompacting).toBe(false);
mockContext.isSubmitting = true;
hook.rerender();
expect(hook.result.current.isCompacting).toBe(false);
});
});

View file

@ -6,6 +6,7 @@ export { default as useTokenLimits } from './useTokenLimits';
export { default as useTokenUsage } from './useTokenUsage';
export { default as useAddedResponse } from './useAddedResponse';
export { default as useChatFunctions } from './useChatFunctions';
export { default as useCompactConversation } from './useCompactConversation';
export { default as useGetAddedConvo } from './useGetAddedConvo';
export { default as useIdChangeEffect } from './useIdChangeEffect';
export { default as useFocusChatEffect } from './useFocusChatEffect';

View file

@ -291,6 +291,7 @@ export default function useChatFunctions({
isRegenerate = false,
isContinued = false,
isEdited = false,
compact = false,
overrideMessages,
overrideFiles,
targetResponseMessageId,
@ -312,9 +313,12 @@ export default function useChatFunctions({
* regenerate of an already-validated file-only turn.
*/
const replayFileCount = overrideFiles?.length ?? 0;
/** A compaction sends no text: it replays the branch, like a regenerate,
* with the response placeholder parented onto the leaf. */
const regenerateShaped = isRegenerate || compact;
if (
!!isSubmitting ||
(!isRegenerate && !isSubmittableMessage(text, (files?.size ?? 0) + replayFileCount))
(!regenerateShaped && !isSubmittableMessage(text, (files?.size ?? 0) + replayFileCount))
) {
return false;
}
@ -398,7 +402,7 @@ export default function useChatFunctions({
let manualSkills = overrideManualSkills;
if (manualSkills == null) {
manualSkills =
isRegenerate || isContinued || isEdited
regenerateShaped || isContinued || isEdited
? []
: drainPendingManualSkills(conversationId ?? Constants.NEW_CONVO);
}
@ -419,7 +423,7 @@ export default function useChatFunctions({
if (quotesSupported) {
if (overrideQuotes != null) {
quotes = overrideQuotes;
} else if (!isRegenerate && !isContinued && !isEdited) {
} else if (!regenerateShaped && !isContinued && !isEdited) {
quotes = drainPendingQuotes(conversationId ?? Constants.NEW_CONVO);
}
}
@ -471,7 +475,7 @@ export default function useChatFunctions({
navigate(`/c/new${projectSearch}`);
}
const targetParentMessageId = isRegenerate ? messageId : latestMessage?.parentMessageId;
const targetParentMessageId = regenerateShaped ? messageId : latestMessage?.parentMessageId;
/**
* If the user regenerated or resubmitted the message, the current parent is technically
* the latest user message, which is passed into `ask`; otherwise, we can rely on the
@ -535,7 +539,9 @@ export default function useChatFunctions({
isCreatedByUser: true,
parentMessageId,
conversationId,
messageId: isContinued && messageId != null && messageId ? messageId : intermediateId,
/** A compaction's "user message" is the leaf itself, so an error lands under it. */
messageId:
(isContinued || compact) && messageId != null && messageId ? messageId : intermediateId,
thread_id,
error: false,
/**
@ -554,7 +560,8 @@ export default function useChatFunctions({
quotes: quotes.length > 0 ? quotes : undefined,
};
const submissionFiles = overrideFiles ?? targetParentMessage?.files;
/** The leaf's files already sit in history; a compaction re-attaches nothing. */
const submissionFiles = compact ? undefined : (overrideFiles ?? targetParentMessage?.files);
const reuseFiles =
(isRegenerate || (overrideFiles != null && overrideFiles.length)) &&
submissionFiles &&
@ -575,7 +582,9 @@ export default function useChatFunctions({
setFiles(new Map());
setFilesToDelete({});
}
} else if (setFiles && files && files.size > 0 && overrideFiles == null) {
} else if (!compact && setFiles && files && files.size > 0 && overrideFiles == null) {
/** A compaction attaches nothing and must not consume files the user
* staged in the composer for their next message. */
// `overrideFiles` (even empty) is authoritative for the submission:
// auto-drained queued messages must never vacuum up attachments the
// user has staged in the composer for their NEXT message.
@ -609,13 +618,14 @@ export default function useChatFunctions({
/** Set only for edited resubmissions; see `TSubmission.editPrefixLength`. */
let editPrefixLength: number | undefined;
const initialResponseId =
responseMessageId ?? `${isRegenerate ? messageId : intermediateId}`.replace(/_+$/, '') + '_';
responseMessageId ??
`${regenerateShaped ? messageId : intermediateId}`.replace(/_+$/, '') + '_';
const initialResponse: TMessage = {
sender: responseSender,
text: '',
endpoint: endpoint ?? '',
parentMessageId: isRegenerate ? messageId : intermediateId,
parentMessageId: regenerateShaped ? messageId : intermediateId,
messageId: initialResponseId,
thread_id,
conversationId,
@ -697,14 +707,14 @@ export default function useChatFunctions({
currentMessages = currentMessages.filter((msg) => msg.messageId !== responseMessageId);
}
const submissionMessages = isRegenerate
const submissionMessages = regenerateShaped
? getRegenerateSubmissionMessages({
messages: currentMessages,
targetResponseMessage,
initialResponseId: initialResponse.messageId,
})
: currentMessages;
const regenerateMessages = isRegenerate ? [...currentMessages] : undefined;
const regenerateMessages = regenerateShaped ? [...currentMessages] : undefined;
logger.log('message_state', initialResponse);
const submission: TSubmission = {
@ -723,7 +733,8 @@ export default function useChatFunctions({
regenerateMessages,
isEdited: isEditOrContinue,
isContinued,
isRegenerate,
isRegenerate: regenerateShaped,
...(compact && { compact: true }),
initialResponse,
isTemporary,
ephemeralAgent,
@ -738,7 +749,7 @@ export default function useChatFunctions({
queuedMessageOrigin: overrideQueuedMessageOrigin,
};
if (isRegenerate) {
if (regenerateShaped) {
setMessages([...submissionMessages, initialResponse]);
focusRegeneratedResponse(initialResponse.parentMessageId);
} else {

View file

@ -0,0 +1,79 @@
import { useCallback, useEffect, useRef } from 'react';
import { atom, useAtom } from 'jotai';
import { Constants, isCompactedLeaf, isAssistantsEndpoint } from 'librechat-data-provider';
import { useLatestMessage } from '~/hooks/Messages/useLatestMessage';
import { useChatContext } from '~/Providers';
/** Conversation whose compaction this client submitted and is still streaming. */
export const compactingConversationAtom = atom<string | null>(null);
/**
* An Assistants thread lives on the provider, so a summary inserted into the
* local history would compact nothing. Callers hide the action there.
*/
export const supportsCompaction = (endpoint?: string | null): boolean =>
endpoint != null && endpoint !== '' && !isAssistantsEndpoint(endpoint);
/**
* Manual context compaction. Submits a summarize-only turn hung off the
* branch's leaf through the ordinary chat pipeline: the summary streams into
* a response placeholder under the leaf, exactly like the automatic detour's,
* and is persisted as the boundary every later turn starts from.
*/
export default function useCompactConversation() {
const { ask, index, conversation, isSubmitting } = useChatContext();
const latestMessage = useLatestMessage(index);
const [compactingConversation, setCompactingConversation] = useAtom(compactingConversationAtom);
const conversationId = conversation?.conversationId;
const hasConversation =
conversationId != null &&
conversationId !== Constants.NEW_CONVO &&
conversationId !== Constants.PENDING_CONVO;
const isCompacting = isSubmitting && compactingConversation === conversationId;
const canCompact =
hasConversation &&
!isSubmitting &&
latestMessage != null &&
latestMessage.parentMessageId != null &&
!isCompactedLeaf(latestMessage);
/** The marker lives from the submission until the turn it started settles:
* cleared on the submitting → idle edge, or on a mount that finds the
* chat already idle (the turn finished while this view was away), never
* on the idle render that precedes the submit. */
const wasSubmitting = useRef(isSubmitting);
const mounted = useRef(false);
useEffect(() => {
const settled = wasSubmitting.current && !isSubmitting;
const mountedIdle = !mounted.current && !isSubmitting;
mounted.current = true;
wasSubmitting.current = isSubmitting;
if ((settled || mountedIdle) && compactingConversation != null) {
setCompactingConversation(null);
}
}, [isSubmitting, compactingConversation, setCompactingConversation]);
const compact = useCallback(() => {
if (!canCompact || latestMessage == null || conversationId == null) {
return;
}
setCompactingConversation(conversationId);
/** The leaf is both the placeholder's parent and the server-side anchor
* (`parentMessageId` is what the controller compacts up to). */
const accepted = ask(
{
text: '',
conversationId,
messageId: latestMessage.messageId,
parentMessageId: latestMessage.messageId,
},
{ compact: true },
);
if (accepted === false) {
setCompactingConversation(null);
}
}, [ask, canCompact, conversationId, latestMessage, setCompactingConversation]);
return { compact, canCompact, isCompacting };
}

View file

@ -383,6 +383,10 @@
"com_error_expired_user_key": "Provided key for {{0}} expired at {{1}}. Please provide a new key and try again.",
"com_error_files_dupe": "Duplicate file detected.",
"com_error_files_empty": "Empty files are not allowed.",
"com_error_compaction_budget": "Could not compact the context: the agent's instructions and tool definitions already exceed the context budget.",
"com_error_compaction_disabled": "Context compaction is not available for this agent because summarization is disabled.",
"com_error_compaction_failed": "Could not compact the context: the summarizer produced no summary. Nothing was changed.",
"com_error_compaction_nothing": "Nothing to compact: the retained recent turns already cover the whole conversation.",
"com_error_context_tokens_detail": "Your messages need {{0}} tokens, but only {{1}} are available.",
"com_error_empty_messages": "Your messages do not fit in the current context window, so there was nothing left to send to the model. Increase the max context size from the conversation parameters, shorten your message, or start a new conversation.",
"com_error_files_process": "An error occurred while processing the file.",
@ -1062,6 +1066,10 @@
"com_ui_connecting": "Connecting",
"com_ui_contact_admin_if_issue_persists": "Contact the Admin if the issue persists",
"com_ui_context": "Context",
"com_ui_context_compact": "Compact context",
"com_ui_context_compact_info": "Replaces the conversation so far with a summary. Earlier messages stay visible here but are no longer sent to the model.",
"com_ui_context_compacted_by_you": "You compacted the context",
"com_ui_context_compacting": "Compacting context...",
"com_ui_context_cost": "Cost",
"com_ui_context_cost_branch": "Cost (this branch)",
"com_ui_context_cost_total": "All branches",

10
package-lock.json generated
View file

@ -64,7 +64,7 @@
"@azure/storage-blob": "^12.30.0",
"@google/genai": "^2.8.0",
"@keyv/redis": "5.1.6",
"@librechat/agents": "^3.8.0",
"@librechat/agents": "^3.8.1",
"@librechat/api": "*",
"@librechat/data-schemas": "*",
"@microsoft/microsoft-graph-client": "^3.0.7",
@ -10636,9 +10636,9 @@
}
},
"node_modules/@librechat/agents": {
"version": "3.8.0",
"resolved": "https://registry.npmjs.org/@librechat/agents/-/agents-3.8.0.tgz",
"integrity": "sha512-d3MGLuxin2RRzOilqE2i9pAMYutKpYVhffm7AE5Q770Ai4Y6+xrHO8vlaRCy7hACUlM92mx1c2553maH9ID3rg==",
"version": "3.8.1",
"resolved": "https://registry.npmjs.org/@librechat/agents/-/agents-3.8.1.tgz",
"integrity": "sha512-77V4K6HI9lQsysbBiMqLxvATaZnMG1+EJPAdIVv47YCFSSku47zPSgEfP+sTaoUpDXy14aGGtmA/kCdIsBaLfg==",
"license": "MIT",
"dependencies": {
"@anthropic-ai/sdk": "^0.115.0",
@ -43645,7 +43645,7 @@
"@azure/storage-blob": "^12.30.0",
"@google/genai": "^2.8.0",
"@keyv/redis": "5.1.6",
"@librechat/agents": "^3.8.0",
"@librechat/agents": "^3.8.1",
"@librechat/data-schemas": "*",
"@modelcontextprotocol/sdk": "^1.30.0",
"@opentelemetry/api": "^1.9.0",

View file

@ -114,7 +114,7 @@
"@azure/storage-blob": "^12.30.0",
"@google/genai": "^2.8.0",
"@keyv/redis": "5.1.6",
"@librechat/agents": "^3.8.0",
"@librechat/agents": "^3.8.1",
"@librechat/data-schemas": "*",
"@modelcontextprotocol/sdk": "^1.30.0",
"@opentelemetry/api": "^1.9.0",

View file

@ -180,6 +180,7 @@ async function callAndCapture(
opts: {
agents?: ReturnType<typeof makeAgent>[];
summarizationConfig?: SummarizationConfig;
summarizeOnly?: boolean;
initialSummary?: { text: string; tokenCount: number };
appConfig?: AppConfig;
messages?: BaseMessage[];
@ -198,6 +199,7 @@ async function callAndCapture(
agents: agents as never,
signal,
summarizationConfig: opts.summarizationConfig,
summarizeOnly: opts.summarizeOnly,
initialSummary: opts.initialSummary,
appConfig: opts.appConfig,
messages: opts.messages,
@ -3672,3 +3674,22 @@ describe('ask_user_question run wiring', () => {
expect(getCheckpointer(config)).toBeDefined();
});
});
// ---------------------------------------------------------------------------
// summarizeOnly resolution (manual compaction)
// ---------------------------------------------------------------------------
describe('summarizeOnly resolution', () => {
it('is absent on an ordinary run', async () => {
const agents = await callAndCapture();
expect(agents[0].summarizeOnly).toBeUndefined();
});
it('marks only the primary agent of a compaction run', async () => {
const agents = await callAndCapture({
agents: [makeAgent({ id: 'agent_primary' }), makeAgent({ id: 'agent_next' })],
summarizeOnly: true,
});
expect(agents[0].summarizeOnly).toBe(true);
expect(agents[1].summarizeOnly).toBeUndefined();
});
});

View file

@ -1563,6 +1563,7 @@ export async function createRun({
indexTokenCountMap,
initialSessions,
summarizationConfig,
summarizeOnly = false,
compactionSemanticIndex,
initialSummary,
modelCallbacks,
@ -1619,6 +1620,12 @@ export async function createRun({
*/
discoveredToolNames?: string[];
summarizationConfig?: SummarizationConfig;
/**
* Manual compaction: the primary agent summarizes the history outright and
* the run ends after the summary without a model call. Applies to the
* primary agent only; a chained or delegated agent never runs.
*/
summarizeOnly?: boolean;
/** Bounded, source-addressed navigation guidance derived with provider messages. */
compactionSemanticIndex?: CompactionSemanticIndex;
/** Cross-run summary from formatAgentMessages, forwarded to AgentContext */
@ -2021,6 +2028,9 @@ export async function createRun({
}
for (const agent of agents) {
const agentInput = buildAgentInput(agent);
if (summarizeOnly && agent === agents[0]) {
agentInput.summarizeOnly = true;
}
const subagentConfigs = buildSubagentConfigs(
agent,
agentInput,

View file

@ -53,3 +53,32 @@ describe('createPayload server URL', () => {
expect(server).toBe(EndpointURLs[EModelEndpoint.assistants]);
});
});
describe('createPayload compaction', () => {
const compactionSubmission = (compact: boolean): TSubmission =>
({
conversation: { conversationId: 'convo-1', endpoint: 'openAI' },
userMessage: { messageId: 'leaf', text: '' },
endpointOption: { endpoint: 'openAI' },
isRegenerate: true,
...(compact ? { compact: true } : {}),
}) as unknown as TSubmission;
it('sends a compaction as such, never as a regenerated user turn', () => {
/** The regenerate shape is a client-side rendering choice (no new user
* bubble); the server keying off `isRegenerate` would rewrite the leaf
* as a user message. */
const { payload } = createPayload(compactionSubmission(true));
expect(payload.compact).toBe(true);
expect(payload.isRegenerate).toBeUndefined();
expect(payload.text).toBe('');
});
it('leaves an ordinary regenerate untouched', () => {
const { payload } = createPayload(compactionSubmission(false));
expect(payload.isRegenerate).toBe(true);
expect('compact' in payload).toBe(false);
});
});

View file

@ -2030,6 +2030,9 @@ export type TStartupConfig = {
langfuseFanoutEnabled?: boolean;
langfuseConnectionAccess?: boolean;
insightsEnabled?: boolean;
/** Manual context compaction, gated by the same `summarization.enabled`
* switch that governs the automatic detour. */
compactionEnabled?: boolean;
interface?: TInterfaceConfig;
turnstile?: TTurnstileConfig;
balance?: TBalanceConfig;
@ -3395,6 +3398,14 @@ export enum ErrorTypes {
* Formatted provider payload exceeded the context budget before invocation
*/
FINAL_CONTEXT_OVERFLOW = 'final_context_overflow',
/**
* A manual compaction the graph could not attempt; `reason` says why
*/
COMPACTION_SKIPPED = 'compaction_skipped',
/**
* A manual compaction whose summarizer produced nothing; history is untouched
*/
COMPACTION_FAILED = 'compaction_failed',
}
/**

View file

@ -19,6 +19,7 @@ export default function createPayload(submission: t.TSubmission) {
isContinued,
isTemporary,
isRegenerate,
compact,
conversation,
editedContent,
ephemeralAgent,
@ -51,7 +52,10 @@ export default function createPayload(submission: t.TSubmission) {
endpoint,
addedConvo,
isTemporary,
isRegenerate,
/** A compaction borrows the regenerate shape client-side only: the server
* must see it as a compaction, never as a regenerated user turn. */
isRegenerate: compact === true ? undefined : isRegenerate,
...(compact === true && { compact: true }),
editedContent,
conversationId,
isContinued: !!(isEdited && isContinued),

View file

@ -1,7 +1,8 @@
import type { ParentMessage } from './messages';
import type { TFile } from './types/files';
import type { TMessage } from './types';
import { buildTree } from './messages';
import { buildTree, isCompactedLeaf } from './messages';
import { ContentTypes } from './types/runs';
const msg = (messageId: string, parentMessageId: string, over: Partial<TMessage> = {}): TMessage =>
({
@ -196,3 +197,26 @@ describe('buildTree', () => {
});
});
});
describe('isCompactedLeaf', () => {
const summary = (overrides: Record<string, unknown> = {}) => ({
type: ContentTypes.SUMMARY,
content: [{ type: ContentTypes.TEXT, text: 'checkpoint' }],
...overrides,
});
it('is true for a bare, finished summary', () => {
expect(isCompactedLeaf({ content: [summary()] } as TMessage)).toBe(true);
});
it.each([
['no content', undefined],
['empty content', []],
['a summary next to text', [summary(), { type: ContentTypes.TEXT, text: 'reply' }]],
['a summary still streaming', [summary({ summarizing: true })]],
['a failed summary', [summary({ failed: true })]],
['an empty summary', [summary({ content: [] })]],
])('is false for %s', (_label, content) => {
expect(isCompactedLeaf({ content } as TMessage)).toBe(false);
});
});

View file

@ -155,3 +155,29 @@ export function buildTree({
}
return tree;
}
/**
* True when a message is a finished manual compaction: every content part is a
* summary and at least one of them carries text. A summary still streaming or
* one that failed does not count, so an interrupted compaction can be retried.
*/
export function isCompactedLeaf(message?: Pick<TMessage, 'content'> | null): boolean {
const content = message?.content;
if (!Array.isArray(content) || content.length === 0) {
return false;
}
let usable = false;
for (const part of content) {
if (part?.type !== ContentTypes.SUMMARY) {
return false;
}
if (part.summarizing === true || part.failed === true) {
continue;
}
const hasText = (part.content ?? []).some(
(block) => typeof block?.text === 'string' && block.text.trim().length > 0,
);
usable = usable || hasText;
}
return usable;
}

View file

@ -133,6 +133,9 @@ export type TPayload = Partial<TMessage> &
Partial<TEndpointOption> & {
isContinued: boolean;
isRegenerate?: boolean;
/** Manual context compaction: summarize the branch up to `parentMessageId`
* and end without a reply. No user message is created. */
compact?: boolean;
conversationId: string | null;
messages?: TMessages;
isTemporary: boolean;
@ -189,6 +192,9 @@ export type TSubmission = {
/** Client-only full message context used to restore branch siblings after scoped regenerate. */
regenerateMessages?: TMessage[];
isRegenerate?: boolean;
/** Manual context compaction; shaped like a regenerate on the client (no new
* user bubble) and sent to the server as a summarize-only turn. */
compact?: boolean;
initialResponse?: TMessage;
conversation: Partial<TConversation>;
endpointOption: TEndpointOption;

View file

@ -718,6 +718,9 @@ export type SummaryContentPart = {
* into this slot are kept, so the renderer needs this to avoid presenting
* truncated text under the "Conversation summarized" label. */
failed?: boolean;
/** Set when the user compacted the context manually rather than the
* automatic detour firing on context pressure. */
initiatedBy?: 'user';
summaryVersion?: number;
model?: string;
provider?: string;