* 🔒 fix: Bound `/files/usage` TTL Hold Instead of Clearing It
`POST /files/usage` marks queued attachments so the 1-hour upload-window
TTL cannot reap them before the client queue drains. It did this by
calling `updateFilesUsage`, which unsets `expiresAt` outright, turning
every touched upload into a permanently retained file.
The client queue is ephemeral browser state, so this also leaks in normal
use: a closed tab or cleared queue leaves nothing referencing the files,
but their TTL is already gone. The same mechanism let an authenticated
user pin arbitrary owned uploads indefinitely, and the route was excluded
from the file limiters, so the touch was entirely unmetered.
Make the operation match its intent, a renewable hold rather than a
release:
- Add `extendFilesTTL`, which pushes `expiresAt` forward by a bounded
window in a single owner-scoped `updateMany`. Two filter guards keep it
safe under client-supplied ids: `$exists: true` so an already-released
file never has a TTL re-added (that would schedule a live file for
deletion), and `$lt` so a hold only ever moves the deadline later.
The owner scope is a required argument, so an unscoped call is a no-op
rather than a cross-user update.
- `handleFilesUsageRequest` now holds for 24h instead of clearing, and no
longer increments `usage`, since a queue touch is not a send. The real
release still happens at drain, where `updateFilesUsage` marks the
files used against an actual message.
- Give `/usage` its own per-user limiter. Keeping it off the upload quota
was intentional, leaving it unmetered was not.
Abandoned queues are now reaped on schedule, and a replayed touch can only
ever re-assert the same bounded window.
* 🔒 fix: Anchor the `/files/usage` hold to upload time
Codex review on b687922.
The hold derived each new deadline from `Date.now()`, so a caller touching
once a day advanced it by another 24h every time, far below the rate limit.
That left indefinite preservation reachable and made the PR's replay claim
wrong: the window was bounded per call but not in aggregate.
Anchor the deadline to the file's immutable `createdAt` instead of the
request clock. `extendFilesTTL` now takes a lifetime and sets
`expiresAt = max(expiresAt, createdAt + holdMs)` in an aggregation
pipeline, so the target is a fixed point per file and replay is inert
rather than merely bounded. `$max` keeps the widen-only property and the
`expiresAt: {$exists: true}` filter still refuses to resurrect a released
TTL; `createdAt: {$exists: true}` fail-closes when the anchor is absent.
The update runs with `timestamps: false`: a hold is TTL bookkeeping, not a
content write, and bumping `updatedAt` also made every re-touch count as a
modification, hiding whether the deadline actually moved.
Also drop four `.node_modules-*` symlinks that `git add -A` swept in from
an npm install. They pointed at absolute paths on one machine, so every
other checkout got dangling entries. Added the pattern to .gitignore so a
workspace install cannot reintroduce them.
* 🔒 fix: Track the configured approval window in the `/files/usage` hold
Codex review on 9277620.
`endpoints.agents.checkpointer.ttl` is a positive int with no upper bound,
and its docs invite raising it for longer review windows. It drives the
pending-action expiry, so a run can legitimately stay paused past 24h. The
fixed 24h lifetime would then let Mongo reap an attachment while its
approval was still live, and the later queue drain would send a file that
no longer exists.
Replace the fixed constant with `resolveFilesUsageHoldMs`, which adds the
configured approval window to a 24h baseline covering upload, enqueue, and
the run reaching its pause. The route reads the window from the same
`getApprovalTtlMs(checkpointerCfg)` the pending action uses, so the two
stay in lockstep.
The replay bound is unaffected: the window is a per-deployment constant and
the deadline is still `createdAt + holdMs`, so a replayed touch re-asserts
the same instant and `$max` skips the write. Only an operator config change
moves it, never a client.
* 🔒 fix: Renew the `/files/usage` hold across queued runs, under a ceiling
Codex review on 2bd3c52.
The drain sends one queued item per run completion, and each item starts a
run that may itself pause for the full approval window. Since the hold was
taken once at enqueue and pinned to the upload time, an item several places
back could sit through multiple approval windows and lose its attachment
while its chip and the live approval were still there. Another regression
from this PR: the old `$unset` made retention permanent, so deep queues
happened to work.
The queue is unbounded, so no fixed lifetime covers it. Split the hold into
a renewable window and a ceiling:
expiresAt = max(expiresAt, min(now + renewMs, createdAt + maxLifetimeMs))
`renewMs` covers one run's wait and is granted from now, so a queue that is
still draining re-asserts it at each transition; `useQueueDrain` now marks
the remaining items' files whenever it pops one. `maxLifetimeMs` is
measured from the immutable upload time and clamps every renewal, so
repeated touches converge on a ceiling instead of advancing per call, which
keeps the replay bound from the previous round intact.
This also tightens abandonment: a queue nobody drains now lapses one
`renewMs` after its last touch instead of surviving to the ceiling.
`useQueueDrain`'s spec gained a QueryClientProvider, since the renewal goes
through react-query.
* 🔒 fix: Renew queued holds on a heartbeat, and stop dropping batches
Codex review on f616bed.
Three gaps in the renewal added last commit:
- `collectQueuedFileIds` returned early at the server's 10-id cap, so a
remainder holding more than one batch renewed only its first message and
left the rest on their enqueue-time hold. Collect everything and split
into capped requests instead of truncating.
- A refused `ask()` restores the popped item, but renewal ran before the
send and covered only the pre-existing remainder. Since the run-end signal
is already consumed, nothing would touch that item again. Renewal now runs
after `ask` and includes the restored item.
- A single run can interrupt for approval more than once, each pause running
to the configured window, so renewing only at drain transitions leaves a
gap longer than `renewMs` with no renewal in it. The ceiling cannot help
when nothing renews.
The third is the same structural gap as the previous round along a new axis:
renewal tied to discrete events loses the file whenever two events are
further apart than the hold. Rather than hook each transition, renew on a
30 minute heartbeat while anything is queued, which is far below the
smallest hold (24h) and so covers any single gap regardless of cause.
Still bounded: every renewal is clamped against the file's upload time, so
the ceiling is unchanged. A queue nobody has open emits no heartbeat and
lapses one `renewMs` after its last touch, preserving the abandonment
behaviour.
* 🔒 fix: Cover the pre-migration queue, first tick, and `/usage/`
Codex review on 892a27d.
- The heartbeat watched only the active conversation id, but `drainNext`
merges in the `NEW_CONVO` queue, which outlives the URL update: items
queued during the first turn stay keyed there until that run ends. It now
renews the union of both, deduped since they are the same atom before
migration.
- The interval installed without firing, so returning to a conversation
whose hold was nearly up waited out a full period before the first
renewal. It now renews immediately, then on each tick.
- Express's non-strict routing sends `POST /files/usage/` to the same
handler with `req.path === '/usage/'`, so the exact comparison pushed it
onto both upload limiters. A trailing-slash client would have spent its
upload quota, and collected file-upload violations, on metadata
heartbeats. Matching now tolerates the trailing slash.
Firing on effect start also made the drain-time renewal redundant: popping
an item changes the held set, so the renewal effect re-runs on its own. The
one case it cannot see is a refused send, where restoring the item leaves
the set identical, so that branch keeps an explicit renewal and the rest is
removed. Net one request per transition instead of two.
* fix: Add removePorts keyGenerator to all IP-based rate limiters
Six IP-based rate limiters are missing the `keyGenerator: removePorts`
option that is already used by the auth-related limiters (login,
register, resetPassword, verifyEmail). Without it, reverse proxies that
include ports in X-Forwarded-For headers cause
ERR_ERL_INVALID_IP_ADDRESS errors from express-rate-limit.
Fixes#12318
* fix: make removePorts IPv6-safe to prevent rate-limit key collisions
The original regex `/:\d+[^:]*$/` treated the last colon-delimited
segment of bare IPv6 addresses as a port, mangling valid IPs
(e.g. `::1` → `::`, `2001:db8::1` → `2001:db8::`). Distinct IPv6
clients could collapse into the same rate-limit bucket.
Use `net.isIP()` as a fast path for already-valid IPs, then match
bracketed IPv6+port and IPv4+port explicitly. Bare IPv6 addresses
are now returned unchanged.
Also fixes pre-existing property ordering inconsistency in
ttsLimiters.js userLimiterOptions (keyGenerator before store).
* refactor: move removePorts to packages/api as TypeScript, fix import order
- Move removePorts implementation to packages/api/src/utils/removePorts.ts
with proper Express Request typing
- Reduce api/server/utils/removePorts.js to a thin re-export from
@librechat/api for backward compatibility
- Consolidate removePorts import with limiterCache from @librechat/api
in all 6 limiter files, fixing import order (package imports shortest
to longest, local imports longest to shortest)
- Remove narrating inline comments per code style guidelines
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* Refactor: Moved Redis cache infra logic into `packages/api`
- Moved cacheFactory and redisClients from `api/cache` into `packages/api/src/cache` so that features in `packages/api` can use cache without importing backward from the backend.
- Converted all moved files into TS with proper typing.
- Created integration tests to run against actual Redis servers for redisClients and cacheFactory.
- Added a GitHub workflow to run integration tests for the cache feature.
- Bug fix: keyvRedisClient now implements the PING feature properly.
* chore: consolidate imports in getLogStores.js
* chore: reorder imports
* chore: re-add fs-extra as dev dep.
* chore: reorder imports in cacheConfig.ts, cacheFactory.ts, and keyvMongo.ts
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
- Introduced new violation scores for TTS, STT, Fork, Import, and File Upload actions in the .env.example file.
- Updated logViolation function to accept a score parameter, allowing for dynamic severity levels based on the action type.
- Modified limiters for Fork, Import, Message, STT, TTS, Tool Call, and File Upload to utilize the new violation scores when logging violations.
* chore: remove unused redis file
* chore: bump keyv dependencies, and update related imports
* refactor: Implement IoRedis client for rate limiting across middleware, as node-redis via keyv not compatible
* fix: Set max listeners to expected amount
* WIP: memory improvements
* refactor: Simplify getAbortData assignment in createAbortController
* refactor: Update getAbortData to use WeakRef for content management
* WIP: memory improvements in agent chat requests
* refactor: Enhance memory management with finalization registry and cleanup functions
* refactor: Simplify domainParser calls by removing unnecessary request parameter
* refactor: Update parameter types for action tools and agent loading functions to use minimal configs
* refactor: Simplify domainParser tests by removing unnecessary request parameter
* refactor: Simplify domainParser call by removing unnecessary request parameter
* refactor: Enhance client disposal by nullifying additional properties to improve memory management
* refactor: Improve title generation by adding abort controller and timeout handling, consolidate request cleanup
* refactor: Update checkIdleConnections to skip current user when checking for idle connections if passed
* refactor: Update createMCPTool to derive userId from config and handle abort signals
* refactor: Introduce createTokenCounter function and update tokenCounter usage; enhance disposeClient to reset Graph values
* refactor: Update getMCPManager to accept userId parameter for improved idle connection handling
* refactor: Extract logToolError function for improved error handling in AgentClient
* refactor: Update disposeClient to clear handlerRegistry and graphRunnable references in client.run
* refactor: Extract createHandleNewToken function to streamline token handling in initializeClient
* chore: bump @librechat/agents
* refactor: Improve timeout handling in addTitle function for better error management
* refactor: Introduce createFetch instead of using class method
* refactor: Enhance client disposal and request data handling in AskController and EditController
* refactor: Update import statements for AnthropicClient and OpenAIClient to use specific paths
* refactor: Use WeakRef for response handling in SplitStreamHandler to prevent memory leaks
* refactor: Simplify client disposal and rename getReqData to processReqData in AskController and EditController
* refactor: Improve logging structure and parameter handling in OpenAIClient
* refactor: Remove unused GraphEvents and improve stream event handling in AnthropicClient and OpenAIClient
* refactor: Simplify client initialization in AskController and EditController
* refactor: Remove unused mock functions and implement in-memory store for KeyvMongo
* chore: Update dependencies in package-lock.json to latest versions
* refactor: Await token usage recording in OpenAIClient to ensure proper async handling
* refactor: Remove handleAbort route from multiple endpoints and enhance client disposal logic
* refactor: Enhance abort controller logic by managing abortKey more effectively
* refactor: Add newConversation handling in useEventHandlers for improved conversation management
* fix: dropparams
* refactor: Use optional chaining for safer access to request properties in BaseClient
* refactor: Move client disposal and request data processing logic to cleanup module for better organization
* refactor: Remove aborted request check from addTitle function for cleaner logic
* feat: Add Grok 3 model pricing and update tests for new models
* chore: Remove trace warnings and inspect flags from backend start script used for debugging
* refactor: Replace user identifier handling with userId for consistency across controllers, use UserId in clientRegistry
* refactor: Enhance client disposal logic to prevent memory leaks by clearing additional references
* chore: Update @librechat/agents to version 2.4.14 in package.json and package-lock.json
* feat: Implement Redis-based rate limiting, initially import limits
* feat: Enhance rate limiters with Redis support and custom prefixes
* chore: import orders
* chore: update JSDoc for next middleware parameter type in ban and limiter middleware
* feat: add logHeaders middleware to log forwarded headers in requests
* refactor: change log level from info to debug for Redis rate limiters
* feat: increase Redis max listeners and refactor session storage to use Keyv
* feat: verification email
* chore: email verification invalid; localize: update
* fix: redirect to login when signup: fix: save emailVerified correctly
* docs: update ALLOW_UNVERIFIED_EMAIL_LOGIN; fix: don't accept login only when ALLOW_UNVERIFIED_EMAIL_LOGIN = true
* fix: user needs to be authenticated
* style: update
* fix: registration success message and redirect logic
* refactor: use `isEnabled` in ALLOW_UNVERIFIED_EMAIL_LOGIN
* refactor: move checkEmailConfig to server/utils
* refactor: use req as param for verifyEmail function
* chore: jsdoc
* chore: remove console log
* refactor: rename `createNewUser` to `createSocialUser`
* refactor: update typing and add expiresAt field to userSchema
* refactor: begin use of user methods over direct model access for User
* refactor: initial email verification rewrite
* chore: typing
* refactor: registration flow rewrite
* chore: remove help center text
* refactor: update getUser to getUserById and add findUser methods. general fixes from recent changes
* refactor: Update updateUser method to remove expiresAt field and use $set and $unset operations, createUser now returns Id only
* refactor: Update openidStrategy to use optional chaining for avatar check, move saveBuffer init to buffer condition
* refactor: logout on deleteUser mutatation
* refactor: Update openidStrategy login success message format
* refactor: Add emailVerified field to Discord and Facebook profile details
* refactor: move limiters to separate middleware dir
* refactor: Add limiters for email verification and password reset
* refactor: Remove getUserController and update routes and controllers accordingly
* refactor: Update getUserById method to exclude password and version fields
* refactor: move verification to user route, add resend verification option
* refactor: Improve email verification process and resend option
* refactor: remove more direct model access of User and remove unused code
* refactor: replace user authentication methods and token generation
* fix: add user.id to jwt user
* refactor: Update AuthContext to include setError function, add resend link to Login Form, make registration redirect shorter
* fix(updateUserPluginsService): ensure userPlugins variable is defined
* refactor: Delete all shared links for a specific user
* fix: remove use of direct User.save() in handleExistingUser
* fix(importLibreChatConvo): handle missing createdAt field in messages
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
2024-06-07 15:06:47 -04:00
Renamed from api/server/middleware/uploadLimiters.js (Browse further)
