51050cc4d3
* 📑 update mkdocs * rename docker override file and add to gitignore * update .env.example - GOOGLE_MODELS * update index.md * doc refactor: split installation and configuration in two sub-folders * doc update: installation guides * doc update: configuration guides * doc: new docker override guide * doc: new beginner's guide for contributions - Thanks @Berry-13 * doc: update documentation_guidelines.md * doc: update testing.md * doc: update deployment guides * doc: update /dev readme * doc: update general_info * doc: add 0 value to doc weight * doc: add index.md to every doc folders * doc: add weight to index.md and move openrouter from free_ai_apis.md to ai_setup.md * doc: update toc so they display properly on the right had side in mkdocs * doc: update pandoranext.md * doc: index logging_system.md * doc: update readme.md * doc: update litellm.md * doc: update ./dev/readme.md * doc:🔖 new presets.md * doc: minor corrections * doc update: user_auth_system.md and presets.md, doc feat: add mermaid support to mkdocs * doc update: add screenshots to presets.md * doc update: add screenshots to - OpenID with AWS Cognito * doc update: BingAI cookie instruction * doc update: discord auth * doc update: facebook auth * doc: corrections to user_auth_system.md * doc update: github auth * doc update: google auth * doc update: auth clean up * doc organization: installation * doc organization: configuration * doc organization: features+plugins & update:plugins screenshots * doc organization: deploymend + general_info & update: tech_stack.md * doc organization: contributions * doc: minor fixes * doc: minor fixes
2.3 KiB
| title | weight |
|---|---|
| 🪙 Token Usage | -7 |
Token Usage
As of v6.0.0, LibreChat accurately tracks token usage for the OpenAI/Plugins endpoints. This can be viewed in your Database's "Transactions" collection.
In the future, you will be able to toggle viewing how much a conversation has cost you.
Currently, you can limit user token usage by enabling user balances. Set the following .env variable to enable this:
CHECK_BALANCE=true # Enables token credit limiting for the OpenAI/Plugins endpoints
You manually add user balance, or you will need to build out a balance-accruing system for users. This may come as a feature to the app whenever an admin dashboard is introduced.
To manually add balances, run the following command (npm required):
npm run add-balance
You can also specify the email and token credit amount to add, e.g.:
npm run add-balance danny@librechat.ai 1000
This works well to track your own usage for personal use; 1000 credits = $0.001 (1 mill USD)
Notes
- With summarization enabled, you will be blocked from making an API request if the cost of the content that you need to summarize + your messages payload exceeds the current balance
- Counting Prompt tokens is really accurate for OpenAI calls, but not 100% for plugins (due to function calling). It is really close and conservative, meaning its count may be higher by 2-5 tokens.
- The system allows deficits incurred by the completion tokens. It only checks if you have enough for the prompt Tokens, and is pretty lenient with the completion. The graph below details the logic
- The above said, plugins are checked at each generation step, since the process works with multiple API calls. Anything the LLM has generated since the initial user prompt is shared to the user in the error message as seen below.
- There is a 150 token buffer for titling since this is a 2 step process, that averages around 200 total tokens. In the case of insufficient funds, the titling is cancelled before any spend happens and no error is thrown.