mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-16 21:08:17 +00:00
3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
b4afbb3a1e
|
🛝 perf: Render Chats Independently of Catalog Discovery (#17692)
* 🛝 perf: Render Chats Independently of Catalog Discovery
* Preserve Target Authorization and Passive Startup Intent
* Keep Startup Selection, Readiness, and Catalog Keys Consistent
* Preserve Remembered Targets and Validate Scoped Presets
* Retain Remembered Assistant Resource Aliases Across Endpoint Switches
* Apply URL Targets Before Waiting for Send Readiness
* Fix First-Page Agent Discovery and Await Model Selection
* Preserve Spec Access and Refresh Selected Assistant State
* Preserve Builder Deletion State and Start History Reads Earlier
* fix: Preserve Startup Recovery and History Read Ownership
* fix: Retain Express Query Routing in Assistant Lookups
* fix: Prioritize Route Reads and Recover Credential Changes
* fix: Bind Credential Revocation to Its Requested Scope
* fix: Refresh Credential Metadata After Pending Key Reads
* fix: Revalidate Inactive Credential Metadata on Target Mount
* fix: Preserve Reviewed Chat Resolutions After Rebase
* fix: Resolve Dev Conflicts and Preserve Startup Boundaries
* test: Preserve the ChatView Slot Target Boundary
* fix: Defer Terminal Delivery Until Subscription Attachment Completes
* fix: Contain Deferred Terminal Callback Failures
* test: Classify Historical Tool Name Capability Reads
* fix: Activate Requested Catalogs and Isolate Provider Failures
* fix: Keep Healthy Provider Choices Available During Catalog Recovery
* test: Exclude Copied Trace Bundles From Dependency Comparison
---------
Co-authored-by: Lia <lia@librechat.ai>
|
||
|
|
0537930144
|
🗂️ fix: Scope Token Config Cache (#13770)
* fix token config tenant cache scope * fix token config scoped cache backfill * chore sort token config imports |
||
|
|
db7011d567
|
📊 feat: Real-Time Context Window & Token Usage Tracking (#13670)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* 📊 feat: Real-Time Context Window & Token Usage Tracking
* 🧪 fix: Align Pricing Spec Dep Signatures with TxDeps
* 🩹 fix: Resolve Codex Findings for Context Usage Tracking
* 📊 feat: Granular Tool Token Breakdown with Deferred Splits
* 🧪 test: Cover Session Cost in Mock E2E and Scope Usage Selectors
* 🧪 test: Live Host-Pipeline Usage Verification (Env-Gated)
* 🧪 test: Local Real-Provider Multi-Turn E2E Harness
* 🪙 fix: Keep Tagged Usage Buckets Out of the Live Context Estimate
* 🩹 fix: Scoped Token-Config Fallback and Sequential Visibility for Usage Events
* 🩹 fix: Address Usage Review Findings — Cost Timing, Scoped Caches, Finalized Output
- carry the post-snapshot output estimate into the context snapshot at
finalize so the gauge keeps the last response after live resets
- accumulate per-rate billable units and price the session cost at
render, so usage events arriving before the token-config load still
count once it resolves
- pass user-scoped token-config cache keys through loadConfigModels
fetches and drop the controller's unscoped fallback to prevent serving
another user's resolved config
- tag emitted usage events with a per-run seq so resume dedupe never
drops a distinct call with an identical payload
- admit the static tokenConfig override in the custom endpoint schema so
it survives zod parsing into req.config
* 🩹 fix: Align Client Usage Accounting with Backend Cost Semantics
- classify cache tokens by provider (shared inputTokensIncludesCache from
data-provider, consumed by both the backend billing path and the client)
instead of a magnitude heuristic, so Anthropic/Bedrock turns where cache
is smaller than uncached input no longer under-bill input
- mirror resolveCompletionTokens on the client so Vertex-style hidden
thinking tokens are reflected in the Output row and session cost
- prefer endpoint pricing over adapter-provider pricing so a custom
endpoint can price a known model name without built-in rates shadowing it
- carry static cacheRead/cacheWrite overrides through the tokenConfig
schema and buildTokenConfigMap
* 🩹 fix: Honor Static Token Config in Billing; Tighten Usage Freshness
- initializeCustom now uses a static endpoint tokenConfig as the agent's
endpointTokenConfig (billing + balance checks), not just the advertised
UI config — previously the gauge showed admin rates while the agent
billed against built-in tables
- invalidate the token-config query alongside models on user-key add/
revoke so context windows and pricing refresh without a reload
- include maxContextTokens in ChatForm's stabilized conversation memo so
the gauge reflects a changed context-window setting immediately
- feed the live output estimate from the legacy content path (direct and
assistants streams), setting from cumulative part text rather than
accumulating deltas
* 🩹 fix: Resume Usage Dedup, Agent Pricing, and Partial Override Billing
- fold usage events idempotently by (runId, seq) so resume backfill no
longer resets the conversation totals — a mid-stream reconnect keeps the
usage of prompts already completed earlier in the session
- tap replayed pending message/reasoning/content events so output streamed
past the resume snapshot reaches the live estimate, not just the message
- resolve cost against the agent's backing endpoint (Agents conversations
report endpoint `agents` / provider `openAI`, neither of which keys a
custom endpoint's tokenConfig)
- getMultiplier/getCacheMultiplier fall back to the standard tables for
models absent from a partial endpointTokenConfig, so a partial static
override no longer bills non-listed models at defaultRate while the UI
shows the correct pattern rate
* 🩹 fix: Repaired Output in Gauge, Cache-Rate Keys, Config Gate, Usage Cleanup
- live/completed gauge counts the repaired completion (normalized output),
so under-reporting providers don't drop the response from used context
- translate static tokenConfig cacheWrite/cacheRead onto the write/read
keys getCacheMultiplier reads, so cache tokens bill at the configured
rate instead of the prompt-rate fallback
- clear the token index and usage atoms when leaving a conversation, so
visited histories don't accumulate in memory for the tab's lifetime
- wait for startupConfig before mounting the gauge, so a deployment with
contextUsage disabled never briefly mounts it or fires the token-config
query on first load
* 🩹 fix: Move Token-Config Resolution to TS; Key Live Usage by Created Convo
- extract the token-config resolution (override gathering + cache lookup +
buildTokenConfigMap) into resolveTokenConfigMap in packages/api, leaving
the /api controller a thin request-scoped wrapper (CLAUDE.md TS rule)
- getConvoKey prefers the user message's real conversationId once the
`created` event stamps it, so a new chat's first-response live gauge and
totals land under the id TokenUsage subscribes to instead of NEW_CONVO
* 🩹 fix: Clear Stale Redis Job Usage; Live-Tap Legacy Streams; Share Fetched Config
- DEL the Redis job hash before re-creating it so a reused streamId can't
inherit a prior run's contextUsage/tokenUsage and backfill stale usage
- tap the legacy {message,text} stream branch (non-agent OpenAI/Anthropic
streams) into the live estimate, not just the content path
- copy a deduped fetch's token config to every sibling endpoint sharing the
baseURL/key/headers, so /token-config resolves each by its own name
* ⏪ revert: Don't DEL Redis job hash in createJob (breaks cross-replica resume)
createJob is an idempotent join — a second replica calls it for the same
streamId to share an in-flight stream's state. DELeting the hash wiped the
prior replica's persisted created/usage state, so a joining replica missed
the created event (GenerationJobManager cross-replica integration test).
Reverts the F1 change from
|