* 🔒 fix: Bound `/files/usage` TTL Hold Instead of Clearing It
`POST /files/usage` marks queued attachments so the 1-hour upload-window
TTL cannot reap them before the client queue drains. It did this by
calling `updateFilesUsage`, which unsets `expiresAt` outright, turning
every touched upload into a permanently retained file.
The client queue is ephemeral browser state, so this also leaks in normal
use: a closed tab or cleared queue leaves nothing referencing the files,
but their TTL is already gone. The same mechanism let an authenticated
user pin arbitrary owned uploads indefinitely, and the route was excluded
from the file limiters, so the touch was entirely unmetered.
Make the operation match its intent, a renewable hold rather than a
release:
- Add `extendFilesTTL`, which pushes `expiresAt` forward by a bounded
window in a single owner-scoped `updateMany`. Two filter guards keep it
safe under client-supplied ids: `$exists: true` so an already-released
file never has a TTL re-added (that would schedule a live file for
deletion), and `$lt` so a hold only ever moves the deadline later.
The owner scope is a required argument, so an unscoped call is a no-op
rather than a cross-user update.
- `handleFilesUsageRequest` now holds for 24h instead of clearing, and no
longer increments `usage`, since a queue touch is not a send. The real
release still happens at drain, where `updateFilesUsage` marks the
files used against an actual message.
- Give `/usage` its own per-user limiter. Keeping it off the upload quota
was intentional, leaving it unmetered was not.
Abandoned queues are now reaped on schedule, and a replayed touch can only
ever re-assert the same bounded window.
* 🔒 fix: Anchor the `/files/usage` hold to upload time
Codex review on b687922.
The hold derived each new deadline from `Date.now()`, so a caller touching
once a day advanced it by another 24h every time, far below the rate limit.
That left indefinite preservation reachable and made the PR's replay claim
wrong: the window was bounded per call but not in aggregate.
Anchor the deadline to the file's immutable `createdAt` instead of the
request clock. `extendFilesTTL` now takes a lifetime and sets
`expiresAt = max(expiresAt, createdAt + holdMs)` in an aggregation
pipeline, so the target is a fixed point per file and replay is inert
rather than merely bounded. `$max` keeps the widen-only property and the
`expiresAt: {$exists: true}` filter still refuses to resurrect a released
TTL; `createdAt: {$exists: true}` fail-closes when the anchor is absent.
The update runs with `timestamps: false`: a hold is TTL bookkeeping, not a
content write, and bumping `updatedAt` also made every re-touch count as a
modification, hiding whether the deadline actually moved.
Also drop four `.node_modules-*` symlinks that `git add -A` swept in from
an npm install. They pointed at absolute paths on one machine, so every
other checkout got dangling entries. Added the pattern to .gitignore so a
workspace install cannot reintroduce them.
* 🔒 fix: Track the configured approval window in the `/files/usage` hold
Codex review on 9277620.
`endpoints.agents.checkpointer.ttl` is a positive int with no upper bound,
and its docs invite raising it for longer review windows. It drives the
pending-action expiry, so a run can legitimately stay paused past 24h. The
fixed 24h lifetime would then let Mongo reap an attachment while its
approval was still live, and the later queue drain would send a file that
no longer exists.
Replace the fixed constant with `resolveFilesUsageHoldMs`, which adds the
configured approval window to a 24h baseline covering upload, enqueue, and
the run reaching its pause. The route reads the window from the same
`getApprovalTtlMs(checkpointerCfg)` the pending action uses, so the two
stay in lockstep.
The replay bound is unaffected: the window is a per-deployment constant and
the deadline is still `createdAt + holdMs`, so a replayed touch re-asserts
the same instant and `$max` skips the write. Only an operator config change
moves it, never a client.
* 🔒 fix: Renew the `/files/usage` hold across queued runs, under a ceiling
Codex review on 2bd3c52.
The drain sends one queued item per run completion, and each item starts a
run that may itself pause for the full approval window. Since the hold was
taken once at enqueue and pinned to the upload time, an item several places
back could sit through multiple approval windows and lose its attachment
while its chip and the live approval were still there. Another regression
from this PR: the old `$unset` made retention permanent, so deep queues
happened to work.
The queue is unbounded, so no fixed lifetime covers it. Split the hold into
a renewable window and a ceiling:
expiresAt = max(expiresAt, min(now + renewMs, createdAt + maxLifetimeMs))
`renewMs` covers one run's wait and is granted from now, so a queue that is
still draining re-asserts it at each transition; `useQueueDrain` now marks
the remaining items' files whenever it pops one. `maxLifetimeMs` is
measured from the immutable upload time and clamps every renewal, so
repeated touches converge on a ceiling instead of advancing per call, which
keeps the replay bound from the previous round intact.
This also tightens abandonment: a queue nobody drains now lapses one
`renewMs` after its last touch instead of surviving to the ceiling.
`useQueueDrain`'s spec gained a QueryClientProvider, since the renewal goes
through react-query.
* 🔒 fix: Renew queued holds on a heartbeat, and stop dropping batches
Codex review on f616bed.
Three gaps in the renewal added last commit:
- `collectQueuedFileIds` returned early at the server's 10-id cap, so a
remainder holding more than one batch renewed only its first message and
left the rest on their enqueue-time hold. Collect everything and split
into capped requests instead of truncating.
- A refused `ask()` restores the popped item, but renewal ran before the
send and covered only the pre-existing remainder. Since the run-end signal
is already consumed, nothing would touch that item again. Renewal now runs
after `ask` and includes the restored item.
- A single run can interrupt for approval more than once, each pause running
to the configured window, so renewing only at drain transitions leaves a
gap longer than `renewMs` with no renewal in it. The ceiling cannot help
when nothing renews.
The third is the same structural gap as the previous round along a new axis:
renewal tied to discrete events loses the file whenever two events are
further apart than the hold. Rather than hook each transition, renew on a
30 minute heartbeat while anything is queued, which is far below the
smallest hold (24h) and so covers any single gap regardless of cause.
Still bounded: every renewal is clamped against the file's upload time, so
the ceiling is unchanged. A queue nobody has open emits no heartbeat and
lapses one `renewMs` after its last touch, preserving the abandonment
behaviour.
* 🔒 fix: Cover the pre-migration queue, first tick, and `/usage/`
Codex review on 892a27d.
- The heartbeat watched only the active conversation id, but `drainNext`
merges in the `NEW_CONVO` queue, which outlives the URL update: items
queued during the first turn stay keyed there until that run ends. It now
renews the union of both, deduped since they are the same atom before
migration.
- The interval installed without firing, so returning to a conversation
whose hold was nearly up waited out a full period before the first
renewal. It now renews immediately, then on each tick.
- Express's non-strict routing sends `POST /files/usage/` to the same
handler with `req.path === '/usage/'`, so the exact comparison pushed it
onto both upload limiters. A trailing-slash client would have spent its
upload quota, and collected file-upload violations, on metadata
heartbeats. Matching now tolerates the trailing slash.
Firing on effect start also made the drain-time renewal redundant: popping
an item changes the held set, so the renewal effect re-runs on its own. The
one case it cannot see is a refused send, where restoring the item leaves
the set identical, so that branch keeps an explicit renewal and the rest is
removed. Net one request per transition instead of two.
728fc1276e