🐢 perf: Collapse Settled Reasoning Text in AnimatedText (#16403)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Docker Dev Images Build / publish (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run

* perf: Collapse Settled Reasoning Text in AnimatedText

AnimatedText re-split and re-classified the entire accumulated reasoning
string on every streamed token and rendered one element per word, making
a long reasoning stream quadratic. Classification is now append-only: the
leading run of words that finished fading is committed as a single text
node and never re-split, and only the unsettled tail is classified.

* perf: Probe the Settled Reasoning Prefix Instead of Rescanning It

The append check compared the whole settled prefix on every token, which
kept the stream quadratic in character comparisons. It now probes the
prefix head and the region at its boundary, and the settled region is
rendered from the current text so a rewrite can never show stale
characters. Adds a long reasoning stream scenario to the mock harness.

* perf: Settle Spaceless Reasoning Runs Before Their Final Segment

A CJK or Thai reasoning stream never reaches whitespace, so the settled
prefix could not advance and the whole run was re-segmented per token.
Inside a spaceless last word everything before its final segment is now
treated as stable; re-segmenting there only changes fade granularity.

* perf: Bound the Unsettled Reasoning Tail by Length

Settling at part boundaries left any run that never reaches whitespace
(unspaced Hangul, a spaceless script without Intl.Segmenter, long URLs)
in the tail, re-classified on every token. The unsettled region is now
capped at 64 characters regardless of segmentation, replacing the
spaceless-specific rule; only non-animated text is ever cut, so a cut
inside a part is invisible. Tests measure the classified length per
render instead of the DOM shape.

* fix: Keep the Settled Reasoning Cut on a Glyph Boundary

The length cap could cut inside a surrogate pair or a combining or ZWJ
sequence, splitting one glyph across two text nodes. The cut now steps
back past continuation code units.

* fix: Cut the Settled Reasoning Prefix at Grapheme Boundaries

Code-unit rules missed emoji modifiers and regional-indicator flags. The
cut now takes the last grapheme boundary from Intl.Segmenter over the
unsettled tail, which starts at a committed cluster boundary and is
bounded by the cap; the code-unit rule remains as the fallback.

* fix: Skip the Reasoning Tail Cap Without a Grapheme Segmenter

The code-unit fallback could still split emoji modifiers and flags. With
no Intl.Segmenter there is no safe cut, so the cap is not applied and the
tail stops at the last word, as it did before.

---------

Co-authored-by: Lia <lia@librechat.ai>
This commit is contained in:
Marco Beretta 2026-09-27 04:08:22 +02:00 • committed by GitHub
parent f15c8a0f1b
commit 3c72c3fb4a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 306 additions and 6 deletions

View file

@ -261,4 +261,139 @@ describe('AnimatedText', () => {
);
expect(animated).toEqual(['appended', 'tail']);
});
it('collapses the settled prefix into one text node and only renders the tail per word', () => {
const settle = FADE_DURATION_MS + FADE_STAGGER_MAX_MS + 1;
const base = Array.from({ length: 40 }, (_, i) => `w${i}`).join(' ');
const { container, rerender } = render(<AnimatedText text={base} />);
setTime(settle);
rerender(<AnimatedText text={`${base} next`} />);
setTime(settle * 2);
rerender(<AnimatedText text={`${base} next one`} />);
const [first, ...rest] = Array.from(container.childNodes);
expect(first.nodeType).toBe(Node.TEXT_NODE);
expect(first.textContent).toBe(`${base} `);
expect(rest.length).toBeLessThanOrEqual(2);
const animated = Array.from(container.querySelectorAll('span[data-lc-fade]')).map((span) =>
span.textContent?.trim(),
);
expect(animated).toEqual(['one']);
expect(container.textContent).toBe(`${base} next one`);
});
it('keeps the last word out of the settled prefix so it can keep growing', () => {
const settle = FADE_DURATION_MS + FADE_STAGGER_MAX_MS + 1;
const { container, rerender } = render(<AnimatedText text="alpha beta" />);
setTime(settle);
rerender(<AnimatedText text="alpha beta" />);
setTime(settle * 2);
rerender(<AnimatedText text="alpha betamax" />);
expect(container.firstChild?.textContent).toBe('alpha ');
expect(container.childNodes).toHaveLength(2);
expect(container.textContent).toBe('alpha betamax');
});
it.each([
['a spaceless CJK run', '我们需要先分析这个问题然后给出答案'.repeat(8), '首先考虑'],
['an unsegmented run without whitespace', 'x'.repeat(200), 'yyyy'],
['spaced words', 'word '.repeat(60), 'more '],
])('only classifies a bounded tail per render for %s', (_label, base, appended) => {
const settle = FADE_DURATION_MS + FADE_STAGGER_MAX_MS + 1;
const { container, rerender } = render(<AnimatedText text={base} />);
setTime(settle);
rerender(<AnimatedText text={`${base}${appended}`} />);
setTime(settle * 2);
const execSpy = jest.spyOn(RegExp.prototype, 'exec');
rerender(<AnimatedText text={`${base}${appended}${appended}`} />);
const classified = execSpy.mock.calls.reduce(
(longest, [input], index) =>
execSpy.mock.contexts[index].source === '\\S+\\s*'
? Math.max(longest, input.length)
: longest,
0,
);
execSpy.mockRestore();
expect(classified).toBeGreaterThan(0);
expect(classified).toBeLessThanOrEqual(64 + appended.length * 2);
expect(container.textContent).toBe(`${base}${appended}${appended}`);
});
it.each([
['ZWJ families and combining marks', ['\u{1F468}\u200D\u{1F469}\u200D\u{1F467}', 'e\u0301']],
['skin-tone modifiers', ['\u{1F44D}\u{1F3FD}', '\u{1F44B}\u{1F3FF}']],
['regional-indicator flags', ['\u{1F1FA}\u{1F1F8}', '\u{1F1EF}\u{1F1F5}']],
])('never cuts the settled prefix inside a grapheme cluster: %s', (_label, clusters) => {
const settle = FADE_DURATION_MS + FADE_STAGGER_MAX_MS + 1;
let text = clusters.join('').repeat(40);
const { container, rerender } = render(<AnimatedText text={text} />);
for (let step = 1; step <= 12; step++) {
setTime(settle * step);
text += step % 3 === 0 ? 'x' : clusters[step % clusters.length];
rerender(<AnimatedText text={text} />);
const first = container.firstChild?.textContent ?? '';
expect(first.length).toBeGreaterThan(0);
const units = Array.from(
new Intl.Segmenter(undefined, { granularity: 'grapheme' }).segment(text),
);
expect(units.some((unit) => unit.index === first.length)).toBe(true);
}
expect(container.textContent).toBe(text);
});
it('does not cut inside a spaceless run when Intl.Segmenter is unavailable', () => {
const settle = FADE_DURATION_MS + FADE_STAGGER_MAX_MS + 1;
const segmenter = Intl.Segmenter;
const flags = '\u{1F1FA}\u{1F1F8}\u{1F1EF}\u{1F1F5}';
try {
Object.defineProperty(Intl, 'Segmenter', { value: undefined, configurable: true });
let Isolated: typeof AnimatedText = AnimatedText;
jest.isolateModules(() => {
jest.doMock('react', () => React);
Isolated = jest.requireActual<typeof import('../animate')>('../animate').AnimatedText;
});
let text = `a ${flags.repeat(20)}`;
const { container, rerender } = render(<Isolated text={text} />);
for (let step = 1; step <= 6; step++) {
setTime(settle * step);
text += step % 2 === 0 ? 'x' : flags;
rerender(<Isolated text={text} />);
const settledLength = container.firstChild?.textContent?.length ?? 0;
expect(settledLength).toBeLessThanOrEqual(2);
}
expect(container.textContent).toBe(text);
} finally {
Object.defineProperty(Intl, 'Segmenter', { value: segmenter, configurable: true });
}
});
it('re-classifies from the start when the text no longer extends the settled prefix', () => {
const settle = FADE_DURATION_MS + FADE_STAGGER_MAX_MS + 1;
const { container, rerender } = render(<AnimatedText text="first draft here" />);
setTime(settle);
rerender(<AnimatedText text="first draft here now" />);
setTime(settle * 2);
rerender(<AnimatedText text="replaced text" />);
expect(container.textContent).toBe('replaced text');
});
it('renders the current characters when a rewrite keeps the probed prefix regions', () => {
const settle = FADE_DURATION_MS + FADE_STAGGER_MAX_MS + 1;
const head = 'h'.repeat(40);
const edge = 'e'.repeat(40);
const original = `${head} ${'a'.repeat(30)} ${edge} tail`;
const rewritten = `${head} ${'b'.repeat(30)} ${edge} tail more`;
const { container, rerender } = render(<AnimatedText text={original} />);
setTime(settle);
rerender(<AnimatedText text={`${original} next`} />);
setTime(settle * 2);
rerender(<AnimatedText text={rewritten} />);
expect(container.textContent).toBe(rewritten);
});
});

View file

@ -64,17 +64,28 @@ const SPACELESS_REGEX =
/[\u0E00-\u0EFF\u0F00-\u0FFF\u1000-\u109F\u1780-\u17FF\u2E80-\u9FFF\uAC00-\uD7AF\uF900-\uFAFF]/;
let wordSegmenter: Intl.Segmenter | null | undefined;
let graphemeSegmenter: Intl.Segmenter | null | undefined;
function createSegmenter(granularity: 'word' | 'grapheme'): Intl.Segmenter | null {
return typeof Intl !== 'undefined' && typeof Intl.Segmenter === 'function'
? new Intl.Segmenter(undefined, { granularity })
: null;
}
function getWordSegmenter(): Intl.Segmenter | null {
if (wordSegmenter === undefined) {
wordSegmenter =
typeof Intl !== 'undefined' && typeof Intl.Segmenter === 'function'
? new Intl.Segmenter(undefined, { granularity: 'word' })
: null;
wordSegmenter = createSegmenter('word');
}
return wordSegmenter;
}
function getGraphemeSegmenter(): Intl.Segmenter | null {
if (graphemeSegmenter === undefined) {
graphemeSegmenter = createSegmenter('grapheme');
}
return graphemeSegmenter;
}
function pushSegmentedParts(parts: string[], token: string): void {
const segmenter = getWordSegmenter();
if (segmenter == null) {
@ -307,30 +318,110 @@ export function createFadePlugin(hydrated = false): FadePlugin {
}
const DELAY_VAR = '--lc-delay';
const WHITESPACE_REGEX = /\s/;
const PREFIX_PROBE_LENGTH = 32;
/**
* Longest trailing run kept unsettled. Bounds per-token work when the last
* word never reaches whitespace (CJK, Thai, unspaced Hangul, URLs). Only
* settled, non-animated text is cut, and it renders as plain text on both
* sides of the cut, so a cut inside a part is invisible.
*/
const MAX_UNSETTLED_LENGTH = 64;
/**
* Moves a cut offset in `tail` back to a grapheme-cluster boundary, so emoji
* modifier, flag, ZWJ and combining sequences stay whole. `tail` starts at the
* committed settled boundary, which is itself a cluster boundary, and is
* bounded by the cap, so segmenting it stays proportional to the unsettled text.
* Without a grapheme segmenter there is no safe cut, so the cap is not applied.
*/
function glyphBoundary(tail: string, index: number): number {
const segmenter = getGraphemeSegmenter();
if (index <= 0 || segmenter == null) {
return 0;
}
let boundary = 0;
for (const segment of segmenter.segment(tail)) {
if (segment.index > index) {
break;
}
boundary = segment.index;
}
return boundary;
}
/** Offset of the last whitespace-delimited word, which appended text can still extend. */
function lastWordStart(value: string): number {
let index = value.length;
while (index > 0 && WHITESPACE_REGEX.test(value[index - 1])) {
index -= 1;
}
while (index > 0 && !WHITESPACE_REGEX.test(value[index - 1])) {
index -= 1;
}
return index;
}
/**
* Whether `text` still extends the settled `prefix`. Probes only the prefix's
* head and the region at its boundary so the per-token check stays constant
* rather than rescanning the whole accumulated prefix; an append-only stream
* always passes, and a rewrite that shifts or replaces text fails a probe. The
* settled region is always rendered from the current text, so a rewrite that
* slips past the probes can only affect fade timing, never the characters shown.
*/
function extendsPrefix(text: string, prefix: string): boolean {
if (text.length < prefix.length) {
return false;
}
const probe = Math.min(PREFIX_PROBE_LENGTH, prefix.length);
const boundary = prefix.length - probe;
return (
text.startsWith(prefix.slice(0, probe)) && text.startsWith(prefix.slice(boundary), boundary)
);
}
/**
* Plain-text counterpart of the rehype plugin for non-markdown streamed text
* (reasoning). Renders words in fade spans keyed by character offset; only
* render this while the text is actively streaming and render the raw string
* once settled.
* once settled. Classification is append-only: the leading run of parts that
* finished fading collapses into a single committed text node and is never
* re-split, so each streamed token costs work proportional to the unsettled
* tail rather than the accumulated text. The settled boundary never passes a
* word that is still fading, and stops before the last word (which appended
* text can still extend) unless that word outgrows MAX_UNSETTLED_LENGTH.
*/
export const AnimatedText = memo(function AnimatedText({ text }: { text: string }) {
const stateRef = useRef<FadeState | null>(null);
const settledRef = useRef('');
if (stateRef.current == null) {
stateRef.current = createFadeState();
}
const state = stateRef.current;
const settledLength = extendsPrefix(text, settledRef.current) ? settledRef.current.length : 0;
const settled = text.slice(0, settledLength);
const tail = text.slice(settledLength);
const suppress = state.firstRun && text.length > FADE_HYDRATION_THRESHOLD;
const run = beginRun(state, suppress);
const segments = classifyValue(run, text);
run.count = settledLength;
const segments = classifyValue(run, tail);
stageRun(run);
const limit = Math.max(
settledLength + lastWordStart(tail),
settledLength + glyphBoundary(tail, tail.length - MAX_UNSETTLED_LENGTH),
);
const firstAnimated = segments.find((segment) => segment.animated);
const nextSettledLength = Math.min(limit, firstAnimated?.start ?? limit);
useLayoutEffect(() => {
commitRun(state);
settledRef.current = text.slice(0, nextSettledLength);
});
return (
<>
{settled}
{segments.map((segment) => {
if (!segment.animated) {
return <Fragment key={segment.start}>{segment.value}</Fragment>;

View file

@ -36,6 +36,9 @@ const ASSERT_HISTORY_MARKER = 'E2E_ASSERT_HISTORY:';
const ASSERT_QUOTE_MARKER = 'E2E_ASSERT_QUOTE:';
const REPLY_MARKER = 'E2E_REPLY:';
const THINK_REPLY_MARKER = 'E2E_THINK_REPLY:';
/** A reasoning part long enough that its settled prefix dwarfs the words still fading in. */
const LONG_THINK_REPLY_MARKER = 'E2E_LONG_THINK_REPLY:';
const LONG_THINK_WORDS = 400;
const SLOW_THINK_REPLY_MARKER = 'E2E_SLOW_THINK_REPLY:';
const COUNTED_REPLY_MARKER = 'E2E_COUNTED_REPLY:';
const ORDERED_REPLY_MARKER = 'E2E_ORDERED_REPLY:';
@ -641,6 +644,19 @@ function replyResponses(text) {
};
}
const longThinkName = getMarkerValue(text, LONG_THINK_REPLY_MARKER);
if (longThinkName) {
const words = Array.from(
{ length: LONG_THINK_WORDS },
(_, index) => `r${String(index).padStart(3, '0')}`,
);
return {
responses: [
`<think>E2E long reasoning ${longThinkName} ${words.join(' ')} end</think>\n\nE2E reply ${longThinkName}`,
],
};
}
const countedName = getMarkerValue(text, COUNTED_REPLY_MARKER);
if (countedName) {
const count = (countedReplies.get(countedName) ?? 0) + 1;

View file

@ -0,0 +1,58 @@
import { expect, test } from '@playwright/test';
import {
MOCK_ENDPOINTS,
NEW_CHAT_PATH,
messagesView,
selectMockEndpoint,
sendMessage,
} from '../helpers';
const LONG_THINK_WORDS = 400;
const reasoningWords = Array.from(
{ length: LONG_THINK_WORDS },
(_, index) => `r${String(index).padStart(3, '0')}`,
);
const expectedReasoning = (label: string) =>
`E2E long reasoning ${label} ${reasoningWords.join(' ')} end`;
test.describe('long reasoning stream', () => {
test.beforeEach(async ({ page }) => {
await page.addInitScript(() => {
window.localStorage.setItem('showThinking', 'true');
window.localStorage.setItem('smoothStreaming', 'true');
});
});
test('keeps only the newest reasoning words in fade spans and reveals the full text in order @scenario:long-reasoning-stream-settles-prefix-and-shows-full-text', async ({
page,
}) => {
test.setTimeout(90_000);
const label = `long-think-${Date.now()}`;
await page.goto(NEW_CHAT_PATH, { timeout: 10_000 });
await selectMockEndpoint(page, MOCK_ENDPOINTS[0]);
const response = await sendMessage(page, `E2E_LONG_THINK_REPLY:${label}`);
expect(response.ok()).toBeTruthy();
const assistantMessage = messagesView(page).locator('.message-render').last();
const reasoning = assistantMessage.locator('p', { hasText: `E2E long reasoning ${label}` });
await expect(page.getByRole('button', { name: 'Stop generating' })).toBeVisible({
timeout: 30_000,
});
await expect(reasoning).toContainText('r200', { timeout: 30_000 });
const midStream = await reasoning.evaluate((node) => ({
text: node.textContent ?? '',
fading: node.querySelectorAll('span[data-lc-fade]').length,
children: node.childNodes.length,
}));
expect(expectedReasoning(label).startsWith(midStream.text)).toBe(true);
expect(midStream.fading).toBeLessThan(40);
expect(midStream.children).toBeLessThan(60);
await expect(assistantMessage.getByText(`E2E reply ${label}`)).toBeVisible({
timeout: 60_000,
});
await expect(reasoning).toHaveText(expectedReasoning(label));
});
});