mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-09-01 03:27:01 +00:00
⚡ feat: Immediate Conversation Title Generation (#13395)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
* ⚡ feat: Immediate Conversation Title Generation Generate conversation titles as soon as the request is made (in parallel with the response, from the user's first message) as the new default, fixing the #13318 race where a transient /gen_title 404 left new chats stuck on "New Chat". - Add per-endpoint `titleTiming` ('immediate' | 'final') to baseEndpointSchema; `endpoints.all` acts as the global default, unset = immediate. Resolve via a new `resolveTitleTiming` helper (`all` takes precedence). - Fire title generation in parallel with `sendMessage`; `titleConvo` waits (bounded, abortable) for the agent run and titles from the user input only. Persist after the conversation row exists; defer `disposeClient` until the title settles. - Expose `titleGenerationTiming` via startup config; `useTitleGeneration` fetches eagerly in immediate mode with a bounded 404 retry and never treats a transient 404 as final. Skip title queueing for temporary conversations. - Supersedes #13329 while incorporating its bounded 404-retry. * 🩹 fix: Address Copilot review findings on title timing - Guard against an undefined conversationId in addTitle (skip + warn) so the gen_title cache key can't collide as `userId-undefined` and saveConvo is never called without a conversationId. - Gate the title `useQueries` on `enabled` so no /gen_title request fires while unauthenticated (e.g. after logout) even if the module queue holds IDs. - Drop the stale `conversationId` param from the titleConvo JSDoc. - Add a regression test for the undefined-conversationId guard. * 🧵 fix: Harden immediate-title edge cases from codex review - Cancel in-flight immediate title generation when the request aborts: thread job.abortController.signal through addTitle so pressing Stop on a new chat neither consumes the title model nor surfaces a title for a cancelled turn. - Preserve a locally-applied title when the final SSE event's conversation carries no title yet (built before the title was saved), so long immediate-mode responses no longer revert the chat to "New Chat" until reload. - Guarantee one full post-completion gen_title fetch cycle before giving up, so a `final`-mode title (generated only after the stream ends) is still fetched under a global `immediate` default instead of being stranded. - Add regression tests for the abort propagation and the undefined-conversationId guard. * 🔁 fix: Correct title abort, post-completion refetch, and replacement ordering Follow-up to codex review of the immediate-title fixes: - Use a dedicated title AbortController instead of `job.abortController`. The latter is also aborted by `completeJob` on *successful* completion, which cancelled any title slower than a short response. The title is now cancelled only on a real user Stop or when the stream is replaced; a completed-then- aborted title is discarded (no save, cache cleared) rather than persisted. - Reset (not remove) the post-completion title query: `resetQueries` refetches the mounted observer with a fresh retry budget, whereas `removeQueries` left it stuck in its error state, so the promised post-completion cycle never ran. - Run the job-replacement check before resolving `convoReady`, and on a replaced stream cancel/discard the stale title so a discarded prompt can't persist a title. * 🧷 fix: Tighten title abort ordering and endpoint-level timing resolution Follow-up to codex review: - Abort the title controller before resolving `convoReady` on a stopped turn, so the title task can't resume and persist before the later abort. - Cancel the title and unblock its waits on ANY send failure (not just user aborts): a preflight/quota failure before the run exists otherwise hangs `_waitForRun`, deferring client disposal until the 45s title timeout. - Resolve `titleTiming` for custom endpoints via `getCustomEndpointConfig` (their config lives under `endpoints.custom[]`, not `endpoints[endpoint]`). - Derive the startup `titleGenerationTiming` via `resolveTitleTiming` for the agents endpoint so an endpoint-level `final` (without `endpoints.all`) is honored client-side instead of defaulting to immediate and burning eager gen_title polls. * 🪢 fix: Per-agent title timing and safer abort/replacement handling Follow-up to codex review: - Resolve `titleTiming` from the agent's actual endpoint after initialization, so a per-endpoint `final` override on a custom/provider endpoint backing an (ephemeral) agent is honored instead of always using the `agents` endpoint's value. - Don't preserve a locally-fetched title on a stopped (unfinished) turn: the server cancels and discards that title, so keeping it client-side would diverge from server state and leave the stopped chat titled until reload. - On abort/replacement, only delete the cached title if it still holds THIS task's value — a replacement stream shares the `userId-conversationId` key and may have already cached its own valid title that must not be removed. * 🪞 fix: Mirror AgentClient title-config resolution for titleTiming Per maintainer guidance, keep titleTiming resolution identical to how `AgentClient#titleConvo` already resolves the endpoint config — `endpoints.all` is the intended global override and the agent's actual provider endpoint is used: - Resolve via `endpoints.all ?? endpoints[endpoint] ?? getProviderConfig(endpoint) .customEndpointConfig` (was using `getCustomEndpointConfig` directly). Going through `getProviderConfig` picks up its case-insensitive fallback for normalized provider names (e.g. `openrouter` → `OpenRouter`), so a custom endpoint's `titleTiming` is honored like its other title settings. - Add `titleTiming` to the Azure endpoint schema `.pick()` so `endpoints.azureOpenAI.titleTiming` is no longer silently stripped by Zod. Note: per-endpoint title settings being skipped when `endpoints.all` is present is the existing, intended global-override behavior — not changed here. * 🧪 test: Cover useTitleGeneration effect logic (integration) Adds a deterministic white-box integration test that drives the real hook's React effects with a controllable react-query surface, locking down the stateful decisions that previously had no coverage: - immediate mode fetches a queued conversation while its stream is still active - final mode gates until the stream completes, then becomes eligible - success applies the fetched title to the conversation caches - a 404 while active defers (removeQueries) instead of giving up - a 404 after completion forces a fresh fetch via resetQueries (post-completion remount) * feat: Stream immediate title events * style: Format title SSE handler * test: Preserve data-provider exports in OAuth mock * test: Isolate OAuth route API mock * test: Keep OAuth callback factory capture * fix: Replay streamed title events on resume * fix: Honor agents title timing precedence * style: Format title timing fixes
This commit is contained in:
parent
b45e4aeae5
commit
2ef7bdfbc2
22 changed files with 1437 additions and 52 deletions
149
client/src/data-provider/SSE/__tests__/queries.test.ts
Normal file
149
client/src/data-provider/SSE/__tests__/queries.test.ts
Normal file
|
|
@ -0,0 +1,149 @@
|
|||
/**
|
||||
* `~/utils` re-exports from `@librechat/client`, which pulls in framer-motion (an
|
||||
* external peer not present in the jsdom test env). Provide a minimal mock with the
|
||||
* two symbols `queries.ts` uses. `isNotFoundError` mirrors the real axios-only
|
||||
* implementation in `~/utils/errors`.
|
||||
*/
|
||||
jest.mock('~/utils', () => {
|
||||
const isNotFoundError = (error: unknown): boolean => {
|
||||
if (error != null && typeof error === 'object') {
|
||||
const response = (error as { response?: { status?: number } }).response;
|
||||
return response?.status === 404;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
return { isNotFoundError, updateConvoInAllQueries: jest.fn() };
|
||||
});
|
||||
|
||||
jest.mock('librechat-data-provider', () => ({
|
||||
apiBaseUrl: () => '',
|
||||
QueryKeys: { conversation: 'conversation', activeJobs: 'activeJobs' },
|
||||
request: { get: jest.fn() },
|
||||
dataService: { genTitle: jest.fn(), getActiveJobs: jest.fn() },
|
||||
}));
|
||||
|
||||
jest.mock('@tanstack/react-query', () => ({
|
||||
useQuery: jest.fn(() => ({ data: undefined })),
|
||||
useQueries: jest.fn(() => []),
|
||||
useQueryClient: jest.fn(() => ({ setQueryData: jest.fn(), removeQueries: jest.fn() })),
|
||||
}));
|
||||
|
||||
/** `queries.ts` imports `useGetStartupConfig` from `../Endpoints` (i.e.
|
||||
* `data-provider/Endpoints`); from this test file that resolves to `../../Endpoints`.
|
||||
* Mock it so the module under test does not pull in the full data-provider barrel. */
|
||||
jest.mock('../../Endpoints', () => ({
|
||||
useGetStartupConfig: jest.fn(() => ({ data: undefined })),
|
||||
}));
|
||||
|
||||
import { genTitleQueryKey, queueTitleGeneration } from '../queries';
|
||||
|
||||
/** Build a minimal Axios-shaped error with a given HTTP status. */
|
||||
function makeAxiosError(status: number): Error {
|
||||
const err = new Error(`HTTP ${status}`) as Error & {
|
||||
isAxiosError: boolean;
|
||||
response: { status: number };
|
||||
};
|
||||
err.isAxiosError = true;
|
||||
err.response = { status };
|
||||
return err;
|
||||
}
|
||||
|
||||
describe('genTitleQueryKey', () => {
|
||||
it('returns a two-element tuple with the conversationId', () => {
|
||||
expect(genTitleQueryKey('abc-123')).toEqual(['genTitle', 'abc-123']);
|
||||
});
|
||||
|
||||
it('returns different keys for different conversation IDs', () => {
|
||||
expect(genTitleQueryKey('conv-1')).not.toEqual(genTitleQueryKey('conv-2'));
|
||||
});
|
||||
});
|
||||
|
||||
describe('queueTitleGeneration', () => {
|
||||
it('runs without throwing for a new conversation ID', () => {
|
||||
expect(() => queueTitleGeneration('new-conv-queue-1')).not.toThrow();
|
||||
});
|
||||
|
||||
it('is safe to call multiple times for the same conversation ID', () => {
|
||||
expect(() => {
|
||||
queueTitleGeneration('new-conv-queue-2');
|
||||
queueTitleGeneration('new-conv-queue-2');
|
||||
}).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* The title-fetch retry policy in `useTitleGeneration`:
|
||||
*
|
||||
* retry: (failureCount, error) => isNotFoundError(error) && failureCount < 3
|
||||
* retryDelay: () => 5_000
|
||||
*
|
||||
* The server `/gen_title` route waits up to ~15.5s before returning 404 while the
|
||||
* title is still generating. Retrying ONLY on 404 (never on 401/403/5xx/network)
|
||||
* means a transient "still generating" response is never treated as final (#13318),
|
||||
* while genuine errors stay terminal.
|
||||
*
|
||||
* These tests pin the classification contract and the failure cap so any future
|
||||
* change to either is caught.
|
||||
*/
|
||||
describe('title fetch retry policy — error classification', () => {
|
||||
const isNotFoundError = (error: unknown): boolean => {
|
||||
if (error != null && typeof error === 'object') {
|
||||
const response = (error as { response?: { status?: number } }).response;
|
||||
return response?.status === 404;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
it('returns true for a 404 (server still generating the title)', () => {
|
||||
expect(isNotFoundError(makeAxiosError(404))).toBe(true);
|
||||
});
|
||||
|
||||
it('returns false for a 401 (auth failure — do not retry)', () => {
|
||||
expect(isNotFoundError(makeAxiosError(401))).toBe(false);
|
||||
});
|
||||
|
||||
it('returns false for a 500 (server failure — do not retry)', () => {
|
||||
expect(isNotFoundError(makeAxiosError(500))).toBe(false);
|
||||
});
|
||||
|
||||
it('returns false for a plain network Error', () => {
|
||||
expect(isNotFoundError(new Error('Network Error'))).toBe(false);
|
||||
});
|
||||
|
||||
it('returns false for null/undefined', () => {
|
||||
expect(isNotFoundError(null)).toBe(false);
|
||||
expect(isNotFoundError(undefined)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('title fetch retry policy — failure cap', () => {
|
||||
const isNotFoundError = (error: unknown): boolean => {
|
||||
if (error != null && typeof error === 'object') {
|
||||
const response = (error as { response?: { status?: number } }).response;
|
||||
return response?.status === 404;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
const retryPredicate = (failureCount: number, error: unknown): boolean =>
|
||||
isNotFoundError(error) && failureCount < 3;
|
||||
|
||||
const notFound = makeAxiosError(404);
|
||||
|
||||
it('retries the first three 404 failures', () => {
|
||||
expect(retryPredicate(0, notFound)).toBe(true);
|
||||
expect(retryPredicate(1, notFound)).toBe(true);
|
||||
expect(retryPredicate(2, notFound)).toBe(true);
|
||||
});
|
||||
|
||||
it('stops after the third 404 failure', () => {
|
||||
expect(retryPredicate(3, notFound)).toBe(false);
|
||||
});
|
||||
|
||||
it('never retries a non-404 regardless of attempt count', () => {
|
||||
const authErr = makeAxiosError(401);
|
||||
for (let i = 0; i < 5; i++) {
|
||||
expect(retryPredicate(i, authErr)).toBe(false);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,178 @@
|
|||
/**
|
||||
* White-box integration test for `useTitleGeneration`'s effect logic.
|
||||
*
|
||||
* Complements the helper unit tests in `queries.test.ts` by driving the REAL
|
||||
* hook (real React state/effects) with a controllable react-query surface, so
|
||||
* the stateful decisions — immediate-vs-final eligibility, success application,
|
||||
* defer-while-active, and the post-completion `resetQueries` remount — are
|
||||
* deterministically locked down without timer/async flakiness.
|
||||
*
|
||||
* `~/utils` re-exports from `@librechat/client` (framer-motion peer, absent in
|
||||
* jsdom); mocked to the two symbols the hook uses. react-query is mocked so we
|
||||
* control `activeJobIds`, the per-conversation query results, and spy the
|
||||
* QueryClient — the hook's own React effects still run for real.
|
||||
*/
|
||||
let mockActiveJobIds: string[] = [];
|
||||
let mockTiming: 'immediate' | 'final' = 'immediate';
|
||||
let mockQueriesResults: Array<{
|
||||
isSuccess?: boolean;
|
||||
isError?: boolean;
|
||||
data?: { title: string };
|
||||
error?: unknown;
|
||||
}> = [];
|
||||
let mockCapturedQueries: Array<{ queryKey: unknown[] }> = [];
|
||||
|
||||
const mockSetQueryData = jest.fn();
|
||||
const mockRemoveQueries = jest.fn();
|
||||
const mockResetQueries = jest.fn();
|
||||
const mockUpdateConvoInAllQueries = jest.fn();
|
||||
|
||||
jest.mock('@tanstack/react-query', () => ({
|
||||
useQuery: jest.fn(() => ({ data: { activeJobIds: mockActiveJobIds } })),
|
||||
useQueries: jest.fn(({ queries }: { queries: Array<{ queryKey: unknown[] }> }) => {
|
||||
mockCapturedQueries = queries;
|
||||
return mockQueriesResults;
|
||||
}),
|
||||
useQueryClient: jest.fn(() => ({
|
||||
setQueryData: mockSetQueryData,
|
||||
removeQueries: mockRemoveQueries,
|
||||
resetQueries: mockResetQueries,
|
||||
})),
|
||||
}));
|
||||
|
||||
jest.mock('../../Endpoints', () => ({
|
||||
useGetStartupConfig: () => ({ data: { titleGenerationTiming: mockTiming } }),
|
||||
}));
|
||||
|
||||
jest.mock('~/utils', () => ({
|
||||
isNotFoundError: (error: unknown): boolean => {
|
||||
if (error != null && typeof error === 'object') {
|
||||
return (error as { response?: { status?: number } }).response?.status === 404;
|
||||
}
|
||||
return false;
|
||||
},
|
||||
updateConvoInAllQueries: (...args: unknown[]) => mockUpdateConvoInAllQueries(...args),
|
||||
}));
|
||||
|
||||
jest.mock('librechat-data-provider', () => ({
|
||||
apiBaseUrl: () => '',
|
||||
QueryKeys: { conversation: 'conversation', activeJobs: 'activeJobs' },
|
||||
request: { get: jest.fn() },
|
||||
dataService: { genTitle: jest.fn(), getActiveJobs: jest.fn() },
|
||||
}));
|
||||
|
||||
import { renderHook, act } from '@testing-library/react';
|
||||
import {
|
||||
useTitleGeneration,
|
||||
genTitleQueryKey,
|
||||
queueTitleGeneration,
|
||||
markTitleGenerationProcessed,
|
||||
} from '../queries';
|
||||
|
||||
const notFound = { response: { status: 404 } };
|
||||
|
||||
/** queryKeys passed to the latest `useQueries` call (i.e. the ready-to-fetch set). */
|
||||
const eligibleKeys = () => mockCapturedQueries.map((q) => JSON.stringify(q.queryKey));
|
||||
const isEligible = (id: string) => eligibleKeys().includes(JSON.stringify(genTitleQueryKey(id)));
|
||||
|
||||
beforeEach(() => {
|
||||
mockActiveJobIds = [];
|
||||
mockTiming = 'immediate';
|
||||
mockQueriesResults = [];
|
||||
mockCapturedQueries = [];
|
||||
jest.clearAllMocks();
|
||||
});
|
||||
|
||||
describe('useTitleGeneration — eligibility', () => {
|
||||
it('immediate mode: fetches a queued conversation while its stream is still active', () => {
|
||||
mockTiming = 'immediate';
|
||||
mockActiveJobIds = ['conv-imm'];
|
||||
|
||||
renderHook(() => useTitleGeneration(true));
|
||||
act(() => queueTitleGeneration('conv-imm'));
|
||||
|
||||
expect(isEligible('conv-imm')).toBe(true);
|
||||
});
|
||||
|
||||
it('final mode: gates a queued conversation until its stream completes', () => {
|
||||
mockTiming = 'final';
|
||||
mockActiveJobIds = ['conv-fin'];
|
||||
|
||||
const { rerender } = renderHook(() => useTitleGeneration(true));
|
||||
act(() => queueTitleGeneration('conv-fin'));
|
||||
expect(isEligible('conv-fin')).toBe(false);
|
||||
|
||||
// Stream completes — the conversation leaves the active set.
|
||||
mockActiveJobIds = [];
|
||||
rerender();
|
||||
expect(isEligible('conv-fin')).toBe(true);
|
||||
});
|
||||
|
||||
it('stops polling when a title is completed by an SSE event', () => {
|
||||
mockTiming = 'immediate';
|
||||
mockActiveJobIds = ['conv-sse-title'];
|
||||
|
||||
const { rerender } = renderHook(() => useTitleGeneration(true));
|
||||
act(() => queueTitleGeneration('conv-sse-title'));
|
||||
expect(isEligible('conv-sse-title')).toBe(true);
|
||||
|
||||
act(() => markTitleGenerationProcessed('conv-sse-title'));
|
||||
rerender();
|
||||
|
||||
expect(isEligible('conv-sse-title')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('useTitleGeneration — result handling', () => {
|
||||
it('applies the fetched title to the conversation caches on success', () => {
|
||||
mockTiming = 'immediate';
|
||||
mockActiveJobIds = ['conv-ok'];
|
||||
|
||||
const { rerender } = renderHook(() => useTitleGeneration(true));
|
||||
act(() => queueTitleGeneration('conv-ok'));
|
||||
|
||||
mockQueriesResults = [{ isSuccess: true, isError: false, data: { title: 'Quantum Chat' } }];
|
||||
rerender();
|
||||
|
||||
expect(mockSetQueryData).toHaveBeenCalledWith(
|
||||
['conversation', 'conv-ok'],
|
||||
expect.any(Function),
|
||||
);
|
||||
expect(mockUpdateConvoInAllQueries).toHaveBeenCalled();
|
||||
|
||||
const call = mockSetQueryData.mock.calls.find(
|
||||
([key]) => JSON.stringify(key) === JSON.stringify(['conversation', 'conv-ok']),
|
||||
);
|
||||
const updater = call?.[1] as (c?: { title?: string }) => { title?: string };
|
||||
expect(updater({ title: 'New Chat' })).toEqual(
|
||||
expect.objectContaining({ title: 'Quantum Chat' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('a 404 while the stream is active defers (removeQueries), not giving up', () => {
|
||||
mockTiming = 'immediate';
|
||||
mockActiveJobIds = ['conv-active404'];
|
||||
|
||||
const { rerender } = renderHook(() => useTitleGeneration(true));
|
||||
act(() => queueTitleGeneration('conv-active404'));
|
||||
|
||||
mockQueriesResults = [{ isError: true, isSuccess: false, error: notFound }];
|
||||
rerender();
|
||||
|
||||
expect(mockRemoveQueries).toHaveBeenCalledWith(genTitleQueryKey('conv-active404'));
|
||||
expect(mockResetQueries).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('a 404 after the stream completes forces a fresh fetch via resetQueries', () => {
|
||||
mockTiming = 'immediate';
|
||||
mockActiveJobIds = []; // stream already complete
|
||||
|
||||
const { rerender } = renderHook(() => useTitleGeneration(true));
|
||||
act(() => queueTitleGeneration('conv-done404'));
|
||||
|
||||
mockQueriesResults = [{ isError: true, isSuccess: false, error: notFound }];
|
||||
rerender();
|
||||
|
||||
expect(mockResetQueries).toHaveBeenCalledWith(genTitleQueryKey('conv-done404'));
|
||||
});
|
||||
});
|
||||
|
|
@ -2,7 +2,8 @@ import { useEffect, useMemo, useState } from 'react';
|
|||
import { apiBaseUrl, QueryKeys, request, dataService } from 'librechat-data-provider';
|
||||
import { useQuery, useQueries, useQueryClient } from '@tanstack/react-query';
|
||||
import type { Agents, TConversation } from 'librechat-data-provider';
|
||||
import { updateConvoInAllQueries } from '~/utils';
|
||||
import { isNotFoundError, updateConvoInAllQueries } from '~/utils';
|
||||
import { useGetStartupConfig } from '../Endpoints';
|
||||
|
||||
export interface StreamStatusResponse {
|
||||
active: boolean;
|
||||
|
|
@ -45,6 +46,12 @@ export interface ActiveJobsResponse {
|
|||
const titleQueue = new Set<string>();
|
||||
const processedTitles = new Set<string>();
|
||||
|
||||
/** Conversations whose eager (immediate-mode) title fetch 404'd while the stream
|
||||
* was still active. They wait for stream completion before fetching again instead
|
||||
* of busy-looping — covers a per-endpoint `final` override under a global
|
||||
* `immediate` default. */
|
||||
const deferredTitles = new Set<string>();
|
||||
|
||||
/** Listeners to notify when queue changes (for non-resumable streams like assistants) */
|
||||
const queueListeners = new Set<() => void>();
|
||||
|
||||
|
|
@ -56,13 +63,31 @@ export function queueTitleGeneration(conversationId: string) {
|
|||
}
|
||||
}
|
||||
|
||||
export function markTitleGenerationProcessed(conversationId: string) {
|
||||
processedTitles.add(conversationId);
|
||||
titleQueue.delete(conversationId);
|
||||
deferredTitles.delete(conversationId);
|
||||
queueListeners.forEach((listener) => listener());
|
||||
}
|
||||
|
||||
/**
|
||||
* Hook to process the title generation queue.
|
||||
* Only fetches titles AFTER the job completes (not in activeJobIds).
|
||||
*
|
||||
* Timing is driven by the server's effective default (`titleGenerationTiming`):
|
||||
* - `immediate` (default): fetch the title in parallel with the active stream so
|
||||
* it appears while the response is still streaming.
|
||||
* - `final` (legacy): fetch only after the stream completes.
|
||||
*
|
||||
* The title query retries on 404 (server still generating) so a transient
|
||||
* not-ready response is never treated as final (#13318).
|
||||
* Place this high in the component tree (e.g., Nav.tsx).
|
||||
*/
|
||||
export function useTitleGeneration(enabled = true) {
|
||||
const queryClient = useQueryClient();
|
||||
const { data: startupConfig } = useGetStartupConfig();
|
||||
/** Defaults to immediate until startup config loads. */
|
||||
const timing = startupConfig?.titleGenerationTiming ?? 'immediate';
|
||||
|
||||
const [queueVersion, setQueueVersion] = useState(0);
|
||||
const [readyToFetch, setReadyToFetch] = useState<string[]>([]);
|
||||
|
||||
|
|
@ -82,33 +107,54 @@ export function useTitleGeneration(enabled = true) {
|
|||
|
||||
useEffect(() => {
|
||||
const activeSet = new Set(activeJobIds);
|
||||
const completedJobs: string[] = [];
|
||||
const eligible: string[] = [];
|
||||
|
||||
for (const conversationId of titleQueue) {
|
||||
if (!activeSet.has(conversationId) && !processedTitles.has(conversationId)) {
|
||||
completedJobs.push(conversationId);
|
||||
if (processedTitles.has(conversationId)) {
|
||||
continue;
|
||||
}
|
||||
const eager = timing === 'immediate' && !deferredTitles.has(conversationId);
|
||||
if (eager || !activeSet.has(conversationId)) {
|
||||
eligible.push(conversationId);
|
||||
}
|
||||
}
|
||||
|
||||
if (completedJobs.length > 0) {
|
||||
setReadyToFetch((prev) => [...new Set([...prev, ...completedJobs])]);
|
||||
if (eligible.length > 0) {
|
||||
setReadyToFetch((prev) => [...new Set([...prev, ...eligible])]);
|
||||
}
|
||||
}, [activeJobIds, queueVersion]);
|
||||
}, [activeJobIds, queueVersion, timing]);
|
||||
|
||||
// Fetch titles for ready conversations
|
||||
useEffect(() => {
|
||||
setReadyToFetch((prev) => {
|
||||
const next = prev.filter((id) => !processedTitles.has(id));
|
||||
return next.length === prev.length ? prev : next;
|
||||
});
|
||||
}, [queueVersion]);
|
||||
|
||||
// Fetch titles for ready conversations.
|
||||
const titleQueries = useQueries({
|
||||
queries: readyToFetch.map((conversationId) => ({
|
||||
queryKey: genTitleQueryKey(conversationId),
|
||||
queryFn: () => dataService.genTitle({ conversationId }),
|
||||
// Gate on `enabled` so no /gen_title request fires while unauthenticated
|
||||
// (e.g. after logout) even if the module-level queue still holds IDs.
|
||||
enabled,
|
||||
staleTime: Infinity,
|
||||
retry: false,
|
||||
/** Retry only on 404 (title still generating server-side) so a transient
|
||||
* not-ready response is never treated as final. All other errors are
|
||||
* terminal. Bounded retry adapted from PR #13329. */
|
||||
retry: (failureCount: number, error: unknown) => isNotFoundError(error) && failureCount < 3,
|
||||
retryDelay: () => 5_000,
|
||||
})),
|
||||
});
|
||||
|
||||
useEffect(() => {
|
||||
const activeSet = new Set(activeJobIds);
|
||||
titleQueries.forEach((titleQuery, index) => {
|
||||
const conversationId = readyToFetch[index];
|
||||
if (!conversationId || processedTitles.has(conversationId)) return;
|
||||
if (!conversationId || processedTitles.has(conversationId)) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (titleQuery.isSuccess && titleQuery.data) {
|
||||
const { title } = titleQuery.data;
|
||||
|
|
@ -121,17 +167,34 @@ export function useTitleGeneration(enabled = true) {
|
|||
if (window.location.pathname.includes(conversationId)) {
|
||||
document.title = title;
|
||||
}
|
||||
processedTitles.add(conversationId);
|
||||
titleQueue.delete(conversationId);
|
||||
markTitleGenerationProcessed(conversationId);
|
||||
setReadyToFetch((prev) => prev.filter((id) => id !== conversationId));
|
||||
} else if (titleQuery.isError) {
|
||||
// Mark as processed even on error to avoid infinite retries
|
||||
processedTitles.add(conversationId);
|
||||
titleQueue.delete(conversationId);
|
||||
setReadyToFetch((prev) => prev.filter((id) => id !== conversationId));
|
||||
// Retries are exhausted here (the query only retries on 404). A title may
|
||||
// still be generated *after* the stream completes (final mode generates
|
||||
// only once the response ends), so don't treat the first 404 as final —
|
||||
// guarantee one fresh, full-budget fetch cycle that runs post-completion.
|
||||
if (activeSet.has(conversationId)) {
|
||||
// Failed while still streaming: drop and clear so the completion
|
||||
// transition re-promotes a fresh fetch (instead of busy-looping).
|
||||
deferredTitles.add(conversationId);
|
||||
queryClient.removeQueries(genTitleQueryKey(conversationId));
|
||||
setReadyToFetch((prev) => prev.filter((id) => id !== conversationId));
|
||||
} else if (!deferredTitles.has(conversationId)) {
|
||||
// First failure at/after completion without a prior deferral: grant one
|
||||
// fresh cycle. Polling has stopped (no re-promotion), so reset the query
|
||||
// in place — `resetQueries` refetches active observers with a fresh retry
|
||||
// budget, unlike `removeQueries`, which leaves the observer in error state.
|
||||
deferredTitles.add(conversationId);
|
||||
queryClient.resetQueries(genTitleQueryKey(conversationId));
|
||||
} else {
|
||||
// The post-completion fetch also failed — the title is genuinely absent.
|
||||
markTitleGenerationProcessed(conversationId);
|
||||
setReadyToFetch((prev) => prev.filter((id) => id !== conversationId));
|
||||
}
|
||||
}
|
||||
});
|
||||
}, [titleQueries, readyToFetch, queryClient]);
|
||||
}, [titleQueries, readyToFetch, queryClient, activeJobIds]);
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -80,6 +80,8 @@ jest.mock('~/data-provider', () => ({
|
|||
|
||||
const mockErrorHandler = jest.fn();
|
||||
const mockCreatedHandler = jest.fn();
|
||||
const mockStepHandler = jest.fn();
|
||||
const mockTitleHandler = jest.fn();
|
||||
const mockSetIsSubmitting = jest.fn();
|
||||
const mockClearStepMaps = jest.fn();
|
||||
|
||||
|
|
@ -89,7 +91,8 @@ jest.mock('~/hooks/SSE/useEventHandlers', () =>
|
|||
finalHandler: jest.fn(),
|
||||
createdHandler: mockCreatedHandler,
|
||||
attachmentHandler: jest.fn(),
|
||||
stepHandler: jest.fn(),
|
||||
stepHandler: mockStepHandler,
|
||||
titleHandler: mockTitleHandler,
|
||||
contentHandler: jest.fn(),
|
||||
resetContentHandler: jest.fn(),
|
||||
syncStepMessage: jest.fn(),
|
||||
|
|
@ -177,6 +180,8 @@ describe('useResumableSSE - 404 error path', () => {
|
|||
localStorage.clear();
|
||||
mockErrorHandler.mockClear();
|
||||
mockCreatedHandler.mockClear();
|
||||
mockStepHandler.mockClear();
|
||||
mockTitleHandler.mockClear();
|
||||
mockClearStepMaps.mockClear();
|
||||
mockSetIsSubmitting.mockClear();
|
||||
mockSetQueryData.mockClear();
|
||||
|
|
@ -430,6 +435,67 @@ describe('useResumableSSE - 404 error path', () => {
|
|||
unmount();
|
||||
});
|
||||
|
||||
it('routes title stream events to the title handler', async () => {
|
||||
const submission = buildSubmission();
|
||||
const chatHelpers = buildChatHelpers();
|
||||
|
||||
const { unmount } = renderHook(() => useResumableSSE(submission, chatHelpers));
|
||||
|
||||
await act(async () => {
|
||||
await Promise.resolve();
|
||||
});
|
||||
|
||||
const titleEvent = {
|
||||
event: 'title',
|
||||
data: {
|
||||
conversationId: CONV_ID,
|
||||
title: 'Streamed Title',
|
||||
},
|
||||
};
|
||||
const sse = getLastSSE();
|
||||
await act(async () => {
|
||||
sse._emit('message', { data: JSON.stringify(titleEvent) });
|
||||
});
|
||||
|
||||
expect(mockTitleHandler).toHaveBeenCalledWith(titleEvent);
|
||||
expect(mockStepHandler).not.toHaveBeenCalled();
|
||||
unmount();
|
||||
});
|
||||
|
||||
it('replays title events from resume state sync', async () => {
|
||||
const submission = buildSubmission();
|
||||
const chatHelpers = buildChatHelpers();
|
||||
|
||||
const { unmount } = renderHook(() => useResumableSSE(submission, chatHelpers));
|
||||
|
||||
await act(async () => {
|
||||
await Promise.resolve();
|
||||
});
|
||||
|
||||
const titleEvent = {
|
||||
event: 'title',
|
||||
data: {
|
||||
conversationId: CONV_ID,
|
||||
title: 'Resumed Title',
|
||||
},
|
||||
};
|
||||
const sse = getLastSSE();
|
||||
await act(async () => {
|
||||
sse._emit('message', {
|
||||
data: JSON.stringify({
|
||||
sync: true,
|
||||
resumeState: {
|
||||
runSteps: [],
|
||||
titleEvent,
|
||||
},
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
expect(mockTitleHandler).toHaveBeenCalledWith(titleEvent);
|
||||
unmount();
|
||||
});
|
||||
|
||||
it.each([undefined, 500, 503])(
|
||||
'does not call errorHandler for responseCode %s (reconnect path)',
|
||||
async (responseCode) => {
|
||||
|
|
|
|||
|
|
@ -33,7 +33,11 @@ import {
|
|||
removeConvoFromAllQueries,
|
||||
findConversationInInfinite,
|
||||
} from '~/utils';
|
||||
import { startupConfigKey, queueTitleGeneration } from '~/data-provider';
|
||||
import {
|
||||
startupConfigKey,
|
||||
queueTitleGeneration,
|
||||
markTitleGenerationProcessed,
|
||||
} from '~/data-provider';
|
||||
import useAttachmentHandler from '~/hooks/SSE/useAttachmentHandler';
|
||||
import useContentHandler from '~/hooks/SSE/useContentHandler';
|
||||
import useStepHandler from '~/hooks/SSE/useStepHandler';
|
||||
|
|
@ -53,6 +57,17 @@ type TSyncData = {
|
|||
conversationId: string;
|
||||
};
|
||||
|
||||
type TTitleEvent = {
|
||||
event: 'title';
|
||||
data?: {
|
||||
conversationId?: string;
|
||||
title?: string;
|
||||
};
|
||||
};
|
||||
|
||||
const hasRealTitle = (title?: string | null): title is string =>
|
||||
title != null && title !== '' && title !== 'New Chat';
|
||||
|
||||
export type EventHandlerParams = {
|
||||
isAddedRequest?: boolean;
|
||||
setCompleted: React.Dispatch<React.SetStateAction<Set<unknown>>>;
|
||||
|
|
@ -467,6 +482,42 @@ export default function useEventHandlers({
|
|||
],
|
||||
);
|
||||
|
||||
const titleHandler = useCallback(
|
||||
(event: TTitleEvent) => {
|
||||
const { conversationId, title } = event.data ?? {};
|
||||
if (!conversationId || !hasRealTitle(title)) {
|
||||
return;
|
||||
}
|
||||
|
||||
queryClient.setQueryData<TConversation>([QueryKeys.conversation, conversationId], (convo) =>
|
||||
convo ? { ...convo, title } : convo,
|
||||
);
|
||||
updateConvoInAllQueries(queryClient, conversationId, (convo) => ({ ...convo, title }));
|
||||
markTitleGenerationProcessed(conversationId);
|
||||
|
||||
if (location.pathname.includes(conversationId)) {
|
||||
document.title = title;
|
||||
}
|
||||
|
||||
if (setConversation && !isAddedRequest) {
|
||||
setConversation((prevState) => {
|
||||
if (!prevState) {
|
||||
return prevState;
|
||||
}
|
||||
if (prevState.conversationId && prevState.conversationId !== conversationId) {
|
||||
return prevState;
|
||||
}
|
||||
return {
|
||||
...prevState,
|
||||
conversationId,
|
||||
title,
|
||||
};
|
||||
});
|
||||
}
|
||||
},
|
||||
[queryClient, location.pathname, setConversation, isAddedRequest],
|
||||
);
|
||||
|
||||
const finalHandler = useCallback(
|
||||
(data: TFinalResData, submission: EventSubmission) => {
|
||||
const { requestMessage, responseMessage, conversation, runMessages } = data;
|
||||
|
|
@ -523,7 +574,8 @@ export default function useEventHandlers({
|
|||
|
||||
const isNewConvo = conversation.conversationId !== submissionConvo.conversationId;
|
||||
|
||||
if (isNewConvo && conversation.conversationId) {
|
||||
// Skip temporary conversations — the server never generates titles for them.
|
||||
if (isNewConvo && conversation.conversationId && !_isTemporary) {
|
||||
queueTitleGeneration(conversation.conversationId);
|
||||
}
|
||||
|
||||
|
|
@ -600,6 +652,28 @@ export default function useEventHandlers({
|
|||
removeConvoFromAllQueries(queryClient, submissionConvo.conversationId);
|
||||
}
|
||||
|
||||
/** A title applied locally (e.g. an immediate-mode title fetched while the
|
||||
* response was still streaming) must survive the final event, whose
|
||||
* `conversation` was built before the title was saved and so carries no
|
||||
* title yet — otherwise the chat reverts to "New Chat" until reload.
|
||||
* Skip preservation for a stopped (unfinished) turn: the server cancels
|
||||
* and discards that title, so the local one would diverge from server state. */
|
||||
const titlePreservable = responseMessage?.unfinished !== true;
|
||||
const finalConversationId = conversation.conversationId;
|
||||
const shouldRollbackStreamedTitle =
|
||||
!titlePreservable && finalConversationId && !hasRealTitle(serverConversation.title);
|
||||
|
||||
if (shouldRollbackStreamedTitle && finalConversationId) {
|
||||
updateConvoInAllQueries(queryClient, finalConversationId, (convo) => ({
|
||||
...convo,
|
||||
title: null,
|
||||
}));
|
||||
if (location.pathname.includes(finalConversationId)) {
|
||||
const startupConfig = queryClient.getQueryData<TStartupConfig>(startupConfigKey(true));
|
||||
document.title = startupConfig?.appTitle ?? 'LibreChat';
|
||||
}
|
||||
}
|
||||
|
||||
if (setConversation && isAddedRequest !== true) {
|
||||
setConversation((prevState) => {
|
||||
const update = {
|
||||
|
|
@ -609,14 +683,28 @@ export default function useEventHandlers({
|
|||
if (prevState?.model != null && prevState.model !== submissionConvo.model) {
|
||||
update.model = prevState.model;
|
||||
}
|
||||
const prevTitle = prevState?.title;
|
||||
if (titlePreservable && !hasRealTitle(conversation.title) && hasRealTitle(prevTitle)) {
|
||||
update.title = prevTitle;
|
||||
}
|
||||
if (conversation.conversationId) {
|
||||
queryClient.setQueryData<TConversation>(
|
||||
[QueryKeys.conversation, conversation.conversationId],
|
||||
(cachedConvo) =>
|
||||
({
|
||||
(cachedConvo) => {
|
||||
const merged = {
|
||||
...cachedConvo,
|
||||
...serverConversation,
|
||||
}) as TConversation,
|
||||
} as TConversation;
|
||||
const cachedTitle = cachedConvo?.title;
|
||||
if (
|
||||
titlePreservable &&
|
||||
!hasRealTitle(serverConversation.title) &&
|
||||
hasRealTitle(cachedTitle)
|
||||
) {
|
||||
merged.title = cachedTitle;
|
||||
}
|
||||
return merged;
|
||||
},
|
||||
);
|
||||
}
|
||||
return update;
|
||||
|
|
@ -890,6 +978,7 @@ export default function useEventHandlers({
|
|||
messageHandler,
|
||||
contentHandler,
|
||||
createdHandler,
|
||||
titleHandler,
|
||||
syncStepMessage,
|
||||
attachmentHandler,
|
||||
abortConversation,
|
||||
|
|
|
|||
|
|
@ -211,6 +211,7 @@ export default function useResumableSSE(
|
|||
messageHandler,
|
||||
contentHandler,
|
||||
createdHandler,
|
||||
titleHandler,
|
||||
syncStepMessage,
|
||||
attachmentHandler,
|
||||
resetContentHandler,
|
||||
|
|
@ -317,6 +318,11 @@ export default function useResumableSSE(
|
|||
return;
|
||||
}
|
||||
|
||||
if (data.event === 'title') {
|
||||
titleHandler(data);
|
||||
return;
|
||||
}
|
||||
|
||||
if (data.event != null) {
|
||||
stepHandler(data, { ...currentSubmission, userMessage } as EventSubmission);
|
||||
return;
|
||||
|
|
@ -398,11 +404,17 @@ export default function useResumableSSE(
|
|||
}
|
||||
}
|
||||
|
||||
if (data.resumeState?.titleEvent) {
|
||||
titleHandler(data.resumeState.titleEvent);
|
||||
}
|
||||
|
||||
if (data.pendingEvents?.length > 0) {
|
||||
console.log(`[ResumableSSE] Replaying ${data.pendingEvents.length} pending events`);
|
||||
const submission = { ...currentSubmission, userMessage } as EventSubmission;
|
||||
for (const pendingEvent of data.pendingEvents) {
|
||||
if (pendingEvent.event != null) {
|
||||
if (pendingEvent.event === 'title') {
|
||||
titleHandler(pendingEvent);
|
||||
} else if (pendingEvent.event != null) {
|
||||
stepHandler(pendingEvent, submission);
|
||||
} else if (pendingEvent.type != null) {
|
||||
contentHandler({ data: pendingEvent, submission });
|
||||
|
|
@ -670,6 +682,7 @@ export default function useResumableSSE(
|
|||
finalHandler,
|
||||
createdHandler,
|
||||
attachmentHandler,
|
||||
titleHandler,
|
||||
stepHandler,
|
||||
contentHandler,
|
||||
resetContentHandler,
|
||||
|
|
@ -801,9 +814,11 @@ export default function useResumableSSE(
|
|||
setStreamId(newStreamId);
|
||||
// Optimistically add to active jobs
|
||||
addActiveJob(newStreamId);
|
||||
// Queue title generation if this is a new conversation (first message)
|
||||
// Queue title generation if this is a new conversation (first message).
|
||||
// Skip temporary conversations — the server never generates titles for
|
||||
// them, so polling would 404 indefinitely.
|
||||
const isNewConvo = submission.userMessage?.parentMessageId === Constants.NO_PARENT;
|
||||
if (isNewConvo) {
|
||||
if (isNewConvo && !submission.isTemporary) {
|
||||
queueTitleGeneration(newStreamId);
|
||||
}
|
||||
if (isInitialNewConversation(submission)) {
|
||||
|
|
|
|||
|
|
@ -42,6 +42,7 @@ export default function useSSE(
|
|||
messageHandler,
|
||||
contentHandler,
|
||||
createdHandler,
|
||||
titleHandler,
|
||||
attachmentHandler,
|
||||
abortConversation,
|
||||
} = useEventHandlers({
|
||||
|
|
@ -113,6 +114,8 @@ export default function useSSE(
|
|||
};
|
||||
|
||||
createdHandler(data, { ...submission, userMessage } as EventSubmission);
|
||||
} else if (data.event === 'title') {
|
||||
titleHandler(data);
|
||||
} else if (data.event != null) {
|
||||
stepHandler(data, { ...submission, userMessage } as EventSubmission);
|
||||
} else if (data.sync != null) {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue