LibreChat/config/set-balance.js
Danny Avila 696bcb6c1e
🏦 fix: Reserve Credits for In-Flight Requests (#15879)
* 💳 fix: Reserve Balance for In-Flight Requests

Admit each balance-checked request against the credits that other
in-flight requests have not already reserved, and hold its prompt cost
on the balance record until the turn settles. Apply a due auto-refill in
the same fenced write so one refill window credits exactly once.

- data-schemas: reserveBalance / releaseBalanceReservation replace
  createAutoRefillTransaction; the write is a compare-and-swap on
  tokenCredits, reservations and lastRefill (no pipeline updates)
- balance.reservations is select:false so existing reads never see it
- checkBalance returns a reservation handle; BaseClient and the
  assistants controllers release it once the turn settles
- balance.reservationTtlMs bounds how long an unreleased reservation
  counts (default 30 minutes)

* 🧮 refactor: Admit Reservations Without Contention and Record Refills Durably

- admission is a conditional $push/$inc against a reservedCredits total,
  so concurrent admissions of a funded balance commit on their first write
- expired reservations are pruned per element with their credits
- auto-refill is fenced on every setting its decision reads and leaves a
  pendingRefill ledger marker in the same write; any later read records the
  transaction under the fixed id and clears the marker
- a missing record is created with $setOnInsert inside reserveBalance
- reservation lifecycle moves to packages/api (createBalanceReservations,
  withBalanceReservations); BaseClient releases once usage is recorded

* 🪪 fix: Resolve Each User to One Balance Record

- balance records a creator inserts are keyed by the user id, so creators
  racing on a missing record converge on one document; every balance read
  and write resolves a user to their oldest record, so existing duplicates
  stay inert
- upsertBalanceFields takes insert-only fields; the balance-config
  middleware, schedules and lazy initialization pass the starting credit
  through them so it never overwrites a record another writer created
- the auto-refill write is also fenced on reservedCredits
- the scheduled-chat balance pre-skip counts reserved credits as unavailable

* ⏳ fix: Count Only Unexpired Reservations in Scheduled Balance Checks

findBalanceByUser with includeReservedCredits now totals the reservations
that have not expired, so a reservation left by a crashed request stops
pre-skipping scheduled chats at its expiry instead of when a later
reservation write prunes it.

* 🔁 fix: Renew Balance Reservations While Their Request Runs

A reservation is renewed every half TTL until it is released, so only a
reservation whose process stopped expires. Pruning removes a reservation
only while it is still expired at the write, so one renewed after the
pruning read survives.

* 🛟 fix: Keep Reservations Alive Through Renewal Failures and Lost Acks

- renewal reschedules itself: every half TTL, capped to the timer range,
  and a failed renewal retries within a tenth of that window
- an admission write that errors removes its possibly committed
  reservation before rethrowing

* 🛑 fix: Hold a Stopped Turn's Reservation Until Its Charge Lands

- an aborted turn lets its reservations lapse after a minute (or the TTL)
  instead of releasing them, so the Stop request's charge lands before the
  credits are admitted again
- balance.reservationTtlMs has a 10 second minimum, enforced in the schema
  and at runtime, so renewal stays bounded

* ↩️ revert: Release a Stopped Turn's Reservation When the Turn Settles

An aborted turn releases its reservations when it settles, the same as
every other exit, instead of holding them for a fixed lapse window that
refused funded requests after every Stop. The minimum reservation TTL
stays.

* ⏸️ fix: Hold Assistants Reservations Through Background Run Retries

An in-progress Assistants run continues in the background after the
handler responds; the turn's reservations now stay held until that run
settles (createBalanceReservations().holdUntil).

* 🔁 fix: Apply an Auto-Refill Once per Admission

A refill interval that is due again the moment it is applied, such as a zero
interval, let one admission refill repeatedly until the retry bound. Each
admission now applies at most one refill, as the previous check did.

* 🧹 fix: Prune Expired Reservations in One Write

An admission that found expired reservations issued one write per hold, so a
backlog left by a stopped process fanned out into as many concurrent writes.
The prune is now a single write fenced on every hold it removes; a hold renewed
after the read makes the write miss and the admission re-read.

* ⏱️ fix: Renew Reservations From Their Stored Expiry

The next renewal was scheduled half a TTL after the previous write settled,
and the first one half a TTL after the admission returned, so a slow write
could let a live hold expire before its renewal. Renewals are now scheduled
from the expiry the last write stored.

Expired holds are pruned as one bulk write of per-hold fenced updates, so a
hold that changed after the read is skipped while the rest are removed, and
the admission no longer spends a retry on the whole prune.
2026-09-13 12:53:19 -04:00

125 lines
3.5 KiB
JavaScript

const path = require('path');
const mongoose = require('mongoose');
const { getBalanceConfig } = require('@librechat/api');
const { User, Balance } = require('@librechat/data-schemas').createModels(mongoose);
require('module-alias')({ base: path.resolve(__dirname, '..', 'api') });
const { getAppConfig } = require('~/server/services/Config');
const { askQuestion, silentExit } = require('./helpers');
const connect = require('./connect');
(async () => {
await connect();
/**
* Show the welcome / help menu
*/
console.purple('--------------------------');
console.purple('Set balance to a user account!');
console.purple('--------------------------');
/**
* Set up the variables we need and get the arguments if they were passed in
*/
let email = '';
let amount = '';
// If we have the right number of arguments, lets use them
if (process.argv.length >= 3) {
email = process.argv[2];
amount = process.argv[3];
} else {
console.orange('Usage: npm run set-balance <email> <amount>');
console.orange('Note: if you do not pass in the arguments, you will be prompted for them.');
console.purple('--------------------------');
// console.purple(`[DEBUG] Args Length: ${process.argv.length}`);
}
const appConfig = await getAppConfig();
const balanceConfig = getBalanceConfig(appConfig);
if (!balanceConfig?.enabled) {
console.red('Error: Balance is not enabled. Use librechat.yaml to enable it');
silentExit(1);
}
/**
* If we don't have the right number of arguments, lets prompt the user for them
*/
if (!email) {
email = await askQuestion('Email:');
}
// Validate the email
if (!email.includes('@')) {
console.red('Error: Invalid email address!');
silentExit(1);
}
// Validate the user
const user = await User.findOne({ email }).lean();
if (!user) {
console.red('Error: No user with that email was found!');
silentExit(1);
} else {
console.purple(`Found user: ${user.email}`);
}
let balance = await Balance.findOne({ user: user._id }).sort({ _id: 1 }).lean();
if (!balance) {
console.purple('User has no balance!');
} else {
console.purple(`Current Balance: ${balance.tokenCredits}`);
}
if (!amount) {
amount = await askQuestion('amount:');
}
// Validate the amount
if (!amount) {
console.red('Error: Please specify an amount!');
silentExit(1);
}
/**
* Now that we have all the variables we need, lets set the balance
*/
let result;
try {
result =
(await Balance.findOneAndUpdate(
{ user: user._id },
{ tokenCredits: amount },
{ new: true, sort: { _id: 1 } },
).lean()) ??
(await Balance.findOneAndUpdate(
{ _id: user._id },
{ $set: { tokenCredits: amount }, $setOnInsert: { user: user._id } },
{ upsert: true, new: true },
).lean());
} catch (error) {
console.red('Error: ' + error.message);
console.error(error);
silentExit(1);
}
// Check the result
if (result?.tokenCredits == null) {
console.red('Error: Something went wrong while updating the balance!');
console.error(result);
silentExit(1);
}
// Done!
console.green('Balance set successfully!');
console.purple(`New Balance: ${result.tokenCredits}`);
silentExit(0);
})();
process.on('uncaughtException', (err) => {
if (!err.message.includes('fetch failed')) {
console.error('There was an uncaught error:');
console.error(err);
}
if (err.message.includes('fetch failed')) {
return;
} else {
process.exit(1);
}
});