mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-18 05:52:10 +00:00
* 🧷 feat: Carry Ask User Answers Past Compaction The user's answers to `ask_user_question` lived only in the tool result that received them. A checkpoint summary, the tool-output pruner or a full context window could remove that message, and the agent then asked again or acted on a summary's paraphrase. Every answered ask on the conversation branch is now rendered verbatim into the run's dynamic instructions, on the first request and on every resume, where the answer just given is read from the seeded content. Older answers drop first past a token budget; the newest set is always kept. New lever: `endpoints.agents.askUserQuestion.retainedAnswers` (`enabled`, `maxTokens`), on by default with a 4096-token budget. * 🧷 feat: Carry Ask User Answers Past Compaction The user's answers to `ask_user_question` lived only in the tool result that received them. A checkpoint summary, the tool-output pruner or a full context window could remove that message, and the agent then asked again or acted on a summary's paraphrase. Every answered ask on the conversation branch is now quoted verbatim at the top of the current user turn's prompt copy, in the user's own role: rebuilt from the stored rows every turn, never persisted, excluded from the memory copy, and counted in the prompt total with the active encoding. A warm event-actor turn reads the branch on demand. Older answers drop first past a token budget that includes separators; the newest set is always kept. New lever: `endpoints.agents.askUserQuestion.retainedAnswers` (`enabled`, `maxTokens`), on by default with a 4096-token budget. * 🧷 refactor: Read The Warm Branch Inside The Module The warm event-actor read lived as a conditional and a `getMessages` call in the CJS client. The typed module now takes `historyLoaded`, the identity and the query as dependencies, decides when to read, and owns the failure policy for both the read and the block itself; the client passes plumbing only. * 🧷 fix: Complete The Branch Behind A Summary Cut `loadHistory` already stops its walk at a checkpoint summary, so the rows `buildMessages` receives never held the answers this feature carries. The module now detects a branch that stops short of its root, whatever cut it, and completes it through the injected stored-row query, stored rows winning. The block goes on the latest user-authored row so a continued generation keeps it, and the render measures the whole block before reserving the omission note, so a set is only dropped when something truly has to go. * 🧷 fix: Reuse Loaded History And Read Every Ask Stamp A post-compaction turn read the conversation twice: once in `loadHistory`, which cuts at the checkpoint summary, and again in the module to complete the branch. The loader now keeps the rows it read and the module prefers them, querying only for a turn that made no history read. Two stored shapes were missed: a pause record whose `args` is the bare question string, now read as the legacy question, and a call whose input failed validation, whose persisted error output is no longer quoted as an answer. * 🧷 fix: Keep The Block Off The Stored Row And Out Of Memory Copies Array-form content is replaced instead of edited through the array the stored row shares, the block's cost is measured as two fresh counts of the same copy rather than against a possibly calibrated stored count, the memory copy is only built when memory extraction will read it, the loaded rows reach the agents client through a hook and are released once the block exists, and the tool-call-id merge is gone: rows are unique by message id and providers may reuse ids across turns. * 🧷 refactor: Apply The Answers Block Inside The Module The placement lookup, the prepend, the count delta and the memory-copy decision lived in the client after the context kickoff. They now live in `applyRetainedAnswers`, which receives the rows, the prompt copies, the count map, the counter and the memory dependency; the client keeps one call and passes plumbing. * fix: Apply Retained Answers After Summary Replay 🧷 * perf: Count Retained Context Exactly Within Its Budget 🧮 * fix: Preserve Retained Counts Across Checkpoint Recounts 🧾 * fix: Isolate Retained Context Counting and Provenance 🧵 * fix: Replace Warm Retained Context and Verify Answer Provenance 🪪 * fix: Use Workspace-Compatible Array Traversal 🧭 * fix: Refresh Retained Context at the Warm Turn Boundary 🧲
436 lines
15 KiB
JavaScript
436 lines
15 KiB
JavaScript
const mockCreateRun = jest.fn();
|
|
const mockRuntimeCounter = jest.fn((counter) => counter);
|
|
const mockCountFormattedMessageTokens = jest.fn(
|
|
(message) => JSON.stringify(message?.content ?? '').length,
|
|
);
|
|
const mockFormatAgentMessages = jest.fn(() => ({
|
|
messages: [],
|
|
indexTokenCountMap: {},
|
|
summary: undefined,
|
|
boundaryTokenAdjustment: undefined,
|
|
}));
|
|
const mockFormatInstructions = jest.fn().mockResolvedValue('');
|
|
|
|
jest.mock('@librechat/agents', () => ({
|
|
...jest.requireActual('@librechat/agents'),
|
|
createMetadataAggregator: () => ({ handleLLMEnd: jest.fn(), collected: [] }),
|
|
formatAgentMessages: (...args) => mockFormatAgentMessages(...args),
|
|
}));
|
|
|
|
jest.mock('@librechat/api', () => ({
|
|
...jest.requireActual('@librechat/api'),
|
|
checkAccess: jest.fn(),
|
|
withRetainedAnswerTokenCounter: (...args) => mockRuntimeCounter(...args),
|
|
prepareRetainedAnswers: (input) =>
|
|
jest.requireActual('@librechat/api').prepareRetainedAnswers({
|
|
...input,
|
|
countTokens: (text) => mockCountFormattedMessageTokens({ role: 'user', content: text }),
|
|
}),
|
|
createRun: (...args) => mockCreateRun(...args),
|
|
countFormattedMessageTokens: (...args) => mockCountFormattedMessageTokens(...args),
|
|
countTokens: jest.fn((text) => Math.ceil(String(text ?? '').length / 4)),
|
|
createCachedTokenCounter: jest.fn(async () => mockCountFormattedMessageTokens),
|
|
getAgentCheckpointer: jest.fn(),
|
|
hasDurableAgentInterruptCheckpoint: jest.fn().mockResolvedValue(true),
|
|
initializeAgent: jest.fn(),
|
|
isHITLEnabled: jest.fn(() => false),
|
|
loadAgent: jest.fn(),
|
|
maybePrewarmCodeSandbox: jest.fn(),
|
|
}));
|
|
|
|
jest.mock('~/server/services/Config', () => ({ getMCPServerTools: jest.fn() }));
|
|
jest.mock('~/server/services/MCP', () => ({
|
|
resolveConfigServers: jest.fn().mockResolvedValue({}),
|
|
}));
|
|
jest.mock('~/models', () => ({
|
|
bulkInsertTransactions: jest.fn(),
|
|
getCacheMultiplier: jest.fn(),
|
|
getAgent: jest.fn(),
|
|
getMultiplier: jest.fn(),
|
|
getFiles: jest.fn(),
|
|
getMessages: jest.fn(),
|
|
getRoleByName: jest.fn(),
|
|
getUserMemories: jest.fn(),
|
|
getFormattedMemories: jest.fn(),
|
|
isAgentTriggerPrincipalActive: jest.fn().mockResolvedValue(true),
|
|
spendStructuredTokens: jest.fn(),
|
|
spendTokens: jest.fn(),
|
|
updateBalance: jest.fn(),
|
|
}));
|
|
jest.mock('~/config', () => ({
|
|
getMCPManager: jest.fn(() => ({ formatInstructionsForContext: mockFormatInstructions })),
|
|
}));
|
|
|
|
const { Constants, ContentTypes, EModelEndpoint } = require('librechat-data-provider');
|
|
const { getMessages } = require('~/models');
|
|
const AgentClient = require('../client');
|
|
const { applyRetainedAnswers, GenerationJobManager } = jest.requireActual('@librechat/api');
|
|
const { formatAgentMessages } = jest.requireActual('@librechat/agents');
|
|
|
|
async function buildPrompt(client, rows, parent) {
|
|
const built = await client.buildMessages(rows, parent, {});
|
|
const formatted = formatAgentMessages(built.prompt, client.indexTokenCountMap);
|
|
const applied = applyRetainedAnswers({
|
|
block: client.retainedAnswers?.block,
|
|
...formatted,
|
|
tokenCounter: mockCountFormattedMessageTokens,
|
|
});
|
|
return {
|
|
...built,
|
|
prompt: applied.messages,
|
|
counts: applied.indexTokenCountMap,
|
|
memoryMessages: formatted.messages,
|
|
};
|
|
}
|
|
|
|
const ANSWER_LINE = 'Q: Which environment should I deploy to?\nA: staging';
|
|
const LATEST_TEXT = 'what is next?';
|
|
|
|
function askPart(request, output, id) {
|
|
return {
|
|
type: ContentTypes.TOOL_CALL,
|
|
tool_call: {
|
|
id,
|
|
name: 'ask_user_question',
|
|
args: JSON.stringify(request),
|
|
output,
|
|
progress: 1,
|
|
},
|
|
};
|
|
}
|
|
|
|
/** The text a formatted message carries, whether its content is a string or parts. */
|
|
function contentText(message) {
|
|
const content = message?.content;
|
|
if (typeof content === 'string') {
|
|
return content;
|
|
}
|
|
return (content ?? [])
|
|
.filter((part) => part?.type === ContentTypes.TEXT)
|
|
.map((part) => part.text)
|
|
.join('\n');
|
|
}
|
|
|
|
function latestUserMessage() {
|
|
return {
|
|
messageId: 'u3',
|
|
parentMessageId: 'a2',
|
|
sender: 'User',
|
|
text: LATEST_TEXT,
|
|
isCreatedByUser: true,
|
|
};
|
|
}
|
|
|
|
/**
|
|
* A branch whose answered question sits BEFORE a checkpoint summary: the prompt
|
|
* builder's history walk stops at the summary, so the model no longer sees the
|
|
* message that carried the answer.
|
|
*/
|
|
function compactedBranch() {
|
|
return [
|
|
{
|
|
messageId: 'u1',
|
|
parentMessageId: Constants.NO_PARENT,
|
|
sender: 'User',
|
|
text: 'deploy the app',
|
|
isCreatedByUser: true,
|
|
},
|
|
{
|
|
messageId: 'a1',
|
|
parentMessageId: 'u1',
|
|
sender: 'Agent',
|
|
isCreatedByUser: false,
|
|
content: [
|
|
{ type: ContentTypes.TEXT, text: 'Let me check.' },
|
|
askPart(
|
|
{ questions: [{ id: 'environment', question: 'Which environment should I deploy to?' }] },
|
|
JSON.stringify({ answers: { environment: 'staging' } }),
|
|
'tc-1',
|
|
),
|
|
],
|
|
},
|
|
{
|
|
messageId: 'u2',
|
|
parentMessageId: 'a1',
|
|
sender: 'User',
|
|
text: 'go on',
|
|
isCreatedByUser: true,
|
|
},
|
|
{
|
|
messageId: 'a2',
|
|
parentMessageId: 'u2',
|
|
sender: 'Agent',
|
|
isCreatedByUser: false,
|
|
content: [
|
|
{ type: ContentTypes.SUMMARY, text: 'Earlier context, compacted.', tokenCount: 5 },
|
|
{ type: ContentTypes.TEXT, text: 'Deployed.' },
|
|
],
|
|
},
|
|
latestUserMessage(),
|
|
];
|
|
}
|
|
|
|
function makeClient(agentsConfig = {}) {
|
|
const client = new AgentClient({
|
|
req: {
|
|
user: { id: 'user-123' },
|
|
body: { endpoint: EModelEndpoint.agents },
|
|
config: { endpoints: { [EModelEndpoint.agents]: agentsConfig } },
|
|
},
|
|
res: {},
|
|
agent: {
|
|
id: 'agent-123',
|
|
endpoint: EModelEndpoint.openAI,
|
|
provider: EModelEndpoint.openAI,
|
|
instructions: 'Base agent instructions',
|
|
model_parameters: { model: 'gpt-4' },
|
|
tools: [],
|
|
},
|
|
endpoint: EModelEndpoint.agents,
|
|
endpointTokenConfig: {},
|
|
eventHandlers: {},
|
|
contentParts: [],
|
|
collectedUsage: [],
|
|
artifactPromises: [],
|
|
});
|
|
client.conversationId = 'convo-123';
|
|
client.responseMessageId = 'response-123';
|
|
client.shouldSummarize = true;
|
|
client.maxContextTokens = 4096;
|
|
client.recordCollectedUsage = jest.fn().mockResolvedValue();
|
|
return client;
|
|
}
|
|
|
|
describe('AgentClient retained answers', () => {
|
|
afterAll(async () => {
|
|
await GenerationJobManager.destroy();
|
|
});
|
|
|
|
const ROW_QUERY = [
|
|
{ conversationId: 'convo-123', user: 'user-123' },
|
|
'messageId parentMessageId content isCreatedByUser isUserSubmitted userSubmittedPaths userSubmittedMessageFieldPaths',
|
|
];
|
|
|
|
beforeEach(() => {
|
|
jest.clearAllMocks();
|
|
mockFormatInstructions.mockResolvedValue('');
|
|
getMessages.mockResolvedValue([]);
|
|
});
|
|
|
|
it('completes the branch the history read cut at the summary and quotes the answer in the user turn', async () => {
|
|
const client = makeClient();
|
|
client.user = 'user-123';
|
|
client.processMemory = jest.fn();
|
|
const rows = compactedBranch();
|
|
getMessages.mockResolvedValue(rows);
|
|
/** The real loader: one read of the conversation, then the summary-bounded walk. */
|
|
const cut = await client.loadHistory('convo-123', 'u3');
|
|
expect(cut.map((message) => message.messageId)).toEqual(['a2', 'u3']);
|
|
expect(getMessages).toHaveBeenCalledTimes(1);
|
|
expect(client.loadedHistoryRows).toBe(rows);
|
|
|
|
const { prompt, tokenCountMap, counts, memoryMessages } = await buildPrompt(client, cut, 'u3');
|
|
|
|
expect(getMessages).toHaveBeenCalledTimes(1);
|
|
expect(client.loadedHistoryRows).toBeUndefined();
|
|
expect(JSON.stringify(memoryMessages)).not.toContain(ANSWER_LINE);
|
|
const text = prompt.map(contentText).join('\n');
|
|
expect(text).toContain(ANSWER_LINE);
|
|
expect(text.indexOf(ANSWER_LINE)).toBeLessThan(text.indexOf(LATEST_TEXT));
|
|
expect(text.endsWith(LATEST_TEXT)).toBe(true);
|
|
expect(client.options.agent.additional_instructions ?? '').not.toContain(ANSWER_LINE);
|
|
expect(cut[1].text).toBe(LATEST_TEXT);
|
|
expect(cut[1].content).toBeUndefined();
|
|
expect(counts[prompt.length - 1]).toBe(tokenCountMap.u3);
|
|
expect(contentText(memoryMessages[memoryMessages.length - 1])).not.toContain(ANSWER_LINE);
|
|
});
|
|
|
|
it.each(['cold', 'warm'])(
|
|
'delivers the block through %s chatCompletion while memory receives the unmodified SDK transcript',
|
|
async (continuation) => {
|
|
const client = makeClient();
|
|
client.processMemory = jest.fn();
|
|
client.runMemory = jest.fn().mockResolvedValue();
|
|
client.user = 'user-123';
|
|
getMessages.mockResolvedValue(compactedBranch());
|
|
const cut = await client.loadHistory('convo-123', 'u3');
|
|
const built = await client.buildMessages(cut, 'u3', {});
|
|
client.eventActorContinuation = continuation;
|
|
mockFormatAgentMessages.mockImplementationOnce(formatAgentMessages);
|
|
const processStream = jest.fn().mockResolvedValue();
|
|
mockCreateRun.mockResolvedValueOnce({
|
|
Graph: null,
|
|
graphRunnable: { getState: jest.fn().mockResolvedValue({ values: { messages: [] } }) },
|
|
processStream,
|
|
getCalibrationRatio: jest.fn(() => 0),
|
|
getInterrupt: jest.fn(() => undefined),
|
|
});
|
|
|
|
await client.chatCompletion({ payload: built.prompt });
|
|
|
|
expect(mockCreateRun).toHaveBeenCalledTimes(1);
|
|
const input = mockCreateRun.mock.calls[0][0];
|
|
expect(input.messages.map(contentText).join('\n')).toContain(ANSWER_LINE);
|
|
expect(mockRuntimeCounter).toHaveBeenCalledTimes(1);
|
|
expect(input.tokenCounter).toBe(mockRuntimeCounter.mock.results[0].value);
|
|
if (continuation === 'warm') {
|
|
expect(input.indexTokenCountMap).toEqual({});
|
|
expect(
|
|
input.tokenCounter(
|
|
input.messages.find((message) => message.id === 'librechat:retained-answers'),
|
|
),
|
|
).toBeGreaterThan(0);
|
|
} else {
|
|
expect(input.indexTokenCountMap[input.messages.length - 1]).toBe(built.tokenCountMap.u3);
|
|
}
|
|
expect(client.runMemory).toHaveBeenCalledTimes(1);
|
|
expect(contentText(client.runMemory.mock.calls[0][0].at(-1))).not.toContain(ANSWER_LINE);
|
|
expect(client.memoryPayload).toBeNull();
|
|
expect(processStream).toHaveBeenCalledTimes(1);
|
|
expect(processStream.mock.calls[0][0].messages.map(contentText).join('\n')).toContain(
|
|
ANSWER_LINE,
|
|
);
|
|
},
|
|
);
|
|
|
|
it('passes the retained-aware runtime counter to checkpoint resumes', async () => {
|
|
const client = makeClient();
|
|
const job = await GenerationJobManager.createJob('convo-123', 'user-123', 'convo-123');
|
|
client.jobCreatedAt = job.createdAt;
|
|
const counter = jest.fn(() => 123);
|
|
mockRuntimeCounter.mockReturnValueOnce(counter);
|
|
const resume = jest.fn().mockResolvedValue();
|
|
mockCreateRun.mockResolvedValueOnce({
|
|
Graph: null,
|
|
resume,
|
|
processStream: jest.fn().mockResolvedValue(),
|
|
getCalibrationRatio: jest.fn(() => 0),
|
|
getInterrupt: jest.fn(() => undefined),
|
|
});
|
|
|
|
await client.resumeCompletion({
|
|
resumeValue: { answer: 'staging' },
|
|
streamId: 'convo-123',
|
|
checkpointNamespace: 'retained-counter',
|
|
});
|
|
|
|
expect(mockRuntimeCounter).toHaveBeenCalledTimes(1);
|
|
expect(mockCreateRun.mock.calls[0][0]).toEqual(
|
|
expect.objectContaining({ messages: [], tokenCounter: counter }),
|
|
);
|
|
expect(resume).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it('reads nothing when the rows in memory already reach the branch root', async () => {
|
|
const client = makeClient();
|
|
client.shouldSummarize = false;
|
|
const rows = compactedBranch();
|
|
rows[3].content = [{ type: ContentTypes.TEXT, text: 'Deployed.' }];
|
|
|
|
const { prompt } = await buildPrompt(client, rows, 'u3');
|
|
|
|
expect(getMessages).not.toHaveBeenCalled();
|
|
expect(
|
|
contentText(prompt.find((message) => message.id === 'librechat:retained-answers')),
|
|
).toContain(ANSWER_LINE);
|
|
});
|
|
|
|
it('leaves the turn alone when the operator turned retained answers off', async () => {
|
|
const client = makeClient({ askUserQuestion: { retainedAnswers: { enabled: false } } });
|
|
|
|
const { prompt } = await buildPrompt(client, compactedBranch(), 'u3');
|
|
|
|
expect(getMessages).not.toHaveBeenCalled();
|
|
expect(contentText(prompt[prompt.length - 1])).toBe(LATEST_TEXT);
|
|
expect(client.memoryPayload).toBeNull();
|
|
});
|
|
|
|
it('reads the stored branch for a warm event-actor turn that skipped history', async () => {
|
|
const client = makeClient();
|
|
client.eventActorContinuation = 'warm';
|
|
getMessages.mockResolvedValue(compactedBranch());
|
|
|
|
const { prompt } = await buildPrompt(client, [latestUserMessage()], 'u3');
|
|
|
|
expect(getMessages).toHaveBeenCalledWith(...ROW_QUERY);
|
|
expect(
|
|
contentText(prompt.find((message) => message.id === 'librechat:retained-answers')),
|
|
).toContain(ANSWER_LINE);
|
|
});
|
|
|
|
it('leaves an array-form user row and its memory copy untouched', async () => {
|
|
const client = makeClient();
|
|
client.shouldSummarize = false;
|
|
client.processMemory = jest.fn();
|
|
const rows = compactedBranch();
|
|
rows[3].content = [{ type: ContentTypes.TEXT, text: 'Deployed.' }];
|
|
const shared = [{ type: ContentTypes.TEXT, text: LATEST_TEXT }];
|
|
rows[4] = { ...rows[4], text: undefined, content: shared };
|
|
|
|
const { prompt, memoryMessages } = await buildPrompt(client, rows, 'u3');
|
|
|
|
expect(
|
|
contentText(prompt.find((message) => message.id === 'librechat:retained-answers')),
|
|
).toContain(ANSWER_LINE);
|
|
expect(shared).toEqual([{ type: ContentTypes.TEXT, text: LATEST_TEXT }]);
|
|
expect(rows[4].content).toBe(shared);
|
|
expect(contentText(memoryMessages[memoryMessages.length - 1])).toBe(LATEST_TEXT);
|
|
});
|
|
|
|
it('measures the block against a fresh count even when the stored count was calibrated', async () => {
|
|
const client = makeClient();
|
|
client.shouldSummarize = false;
|
|
const rows = compactedBranch();
|
|
rows[3].content = [{ type: ContentTypes.TEXT, text: 'Deployed.' }];
|
|
rows[4].tokenCount = 5000;
|
|
|
|
const { prompt, tokenCountMap, counts } = await buildPrompt(client, rows, 'u3');
|
|
|
|
expect(tokenCountMap.u3).toBe(5000);
|
|
const blockOnly = mockCountFormattedMessageTokens({
|
|
role: 'user',
|
|
content: [{ type: ContentTypes.TEXT, text: ANSWER_LINE }],
|
|
});
|
|
expect(counts[prompt.length - 1]).toBe(5000);
|
|
expect(
|
|
counts[prompt.findIndex((message) => message.id === 'librechat:retained-answers')],
|
|
).toBeGreaterThanOrEqual(blockOnly);
|
|
});
|
|
|
|
it('builds no memory copy when memory processing is inactive', async () => {
|
|
const client = makeClient();
|
|
client.shouldSummarize = false;
|
|
const rows = compactedBranch();
|
|
rows[3].content = [{ type: ContentTypes.TEXT, text: 'Deployed.' }];
|
|
|
|
const { prompt } = await buildPrompt(client, rows, 'u3');
|
|
|
|
expect(
|
|
contentText(prompt.find((message) => message.id === 'librechat:retained-answers')),
|
|
).toContain(ANSWER_LINE);
|
|
expect(client.memoryPayload).toBeNull();
|
|
});
|
|
|
|
it('quotes into the turn being continued when the leaf is the unfinished response', async () => {
|
|
const client = makeClient();
|
|
client.shouldSummarize = false;
|
|
const rows = compactedBranch();
|
|
rows[3].content = [{ type: ContentTypes.TEXT, text: 'Deployed.' }];
|
|
rows.push({
|
|
messageId: 'a3',
|
|
parentMessageId: 'u3',
|
|
sender: 'Agent',
|
|
isCreatedByUser: false,
|
|
content: [{ type: ContentTypes.TEXT, text: 'Next I will' }],
|
|
unfinished: true,
|
|
});
|
|
|
|
const { prompt } = await buildPrompt(client, rows, 'a3');
|
|
|
|
expect(prompt[prompt.length - 1].getType()).toBe('ai');
|
|
expect(
|
|
contentText(prompt.find((message) => message.id === 'librechat:retained-answers')),
|
|
).toContain(ANSWER_LINE);
|
|
expect(contentText(prompt[prompt.length - 1])).not.toContain(ANSWER_LINE);
|
|
});
|
|
});
|