Economics for Inference Engineers
Inference economics is understanding what each generated token costs. Cost per token depends on GPU price per hour, achieved utilisation, batch size and model efficiency. Raising utilisation through batching, quantisation and caching lowers cost, and the build-versus-buy choice weighs self-hosting against hosted APIs.
Stage: Reliability and Cost · Part of the Inference Engineer roadmap · Last reviewed 19 November 2026
What to learn in Economics
- Compute cost per million tokens from GPU price and throughput
- Explain how utilisation drives unit cost
- Compare reserved, on-demand and spot GPU pricing trade-offs
- Evaluate self-hosting against a hosted API for a workload
- Spot optimisations that cut cost without hurting latency targets
Economics interview questions
How do you estimate cost per token for a self-hosted model?
Divide the hourly GPU cost by the tokens produced per hour at realistic utilisation. If a GPU costs a given amount per hour and sustains a measured token rate, the cost per million tokens follows directly. Idle time raises it.
When can self-hosting beat a hosted API?
When traffic is high and steady enough to keep GPUs well utilised, or when you need custom models, data control or latency guarantees. For low or spiky volume, the idle capacity and operations effort often make APIs cheaper.
Where Economics fits in the Inference Engineer roadmap
Economics belongs to the Reliability and Cost stage (advanced level), which takes about 5 hours. Run it for real.
Other units in this stage
Back to the full Inference Engineer roadmap