Status: [LIVE]
Endpoints found:
https://zaid-cdlg.github.io/inference-atlas/?model=meta-llama%2FLlama-3.3-70B-Instruct&use=chat
Source: https://github.com/zaid-cdlg/inference-atlas
# inference-atlas
For popular models the API is often cheaper. inference-atlas shows you where that flips.
**Should you self-host this LLM, or just pay the API?**
inference-atlas is a free calculator you open in your browser. Nothing to install. Pick a model
and a use case, and it shows you:
- **Self-host vs API break-even:** the tokens per day at which running your own GPUs becomes
cheaper than paying for the same model through an API.
- **Model, GPU and precision:** which GPUs fit the model, how many you need, and what FP16, FP8
or INT4 trades away.
- **Users and throughput:** how many people can use it at once, how fast it answers, and the cost
per million tokens.
- **A ready-to-run command:** a `vllm serve` command with the settings filled in.
The model list refreshes every week, so new open-weight models show up on their own.
*Llama 3.3 70B on one NVIDIA B200 (auto-picked, FP8), chat use case with the longest conversation raised to 16k tokens: 68 users fit at 4k tokens each, 17 at 16k. [Open this setup](https://zaid-cdlg.github.io/inference-atlas/?model=meta-llama%2FLlama-3.3-70B-Instruct&use=chat&max_ctx=16384).*
**[Open the calculator](https://zaid-cdlg.github.io/inference-atlas/)** (also on
[Hugging Face](https://huggingface.co/spaces/ZaidCdl