Holds and settlement: why max_tokens is money
One small number in your request decides how many concurrent calls your balance can carry.
Before any paid request goes out, an amount is set aside from your balance. When the answer finishes, the real cost replaces it and the rest is freed. That simple mechanism explains many behaviours that look odd at first.
How the hold is worked out
For chat, the hold is the cost of the prompt plus the output ceiling. The ceiling is your max_tokens; if you send none, the model's maximum output is used.
- An estimate is made and the amount is held from your balance.
- The request goes to the model and the answer comes back.
- The real cost is computed, the hold is released and the real amount is charged.
If you set max_tokens very high, a large amount is held for every request, even when the answer is short.
What it does to concurrency
Parallel requests each take their own hold. Say ten workers run in parallel, each with a large output ceiling. Most of your balance sits in holds and the eleventh request fails, even though real spending is low.
In that case the system behaves plainly:
- If you send no
max_tokensand the balance can't cover the ceiling, that request's limit is lowered to what the balance covers. - If you send
max_tokensand the balance can't cover it, you get a 402, and the message states the number it can cover.
max_tokens for each kind of request. Holds get smaller and you meet insufficient_balance far less often.The cost of a cut-off request
Once the first byte of an answer has arrived, the model can't be changed. If a stream breaks midway, the tokens produced until then are charged. For long answers, keep the received text on your side.
Classes of token
Every token falls into exactly one class: normal input, cache write, cache read, output, reasoning, image and audio. Some models charge a higher rate for very long prompts. Every rate is on each model's page, in rials.
See the real cost0 of 3 steps
- Send a request with
max_tokensof 500 and get a short answer. - See
usage.cost_rialin the response. - In the logs, compare the cost with what was held: only the real amount came off.
What you learned
- The hold is the prompt's cost plus the output ceiling.
- A realistic
max_tokensmakes concurrent work easier. - Next: making repeated prompts cheap.
Keep your place
Sign in to the panel and each lesson and your path are saved, so you continue from the same spot on any device.