Short answer: hosted APIs win on top-end quality, zero operations and low or spiky volumes. Self-hosting wins when data must not leave your environment, when volumes are high and steady, or when a smaller open model is good enough for the task. Many organisations end up hybrid.
Comparison
| Hosted API | Self-hosted open model | |
|---|---|---|
| Quality ceiling | Highest (frontier models) | Very good; close for many tasks |
| Privacy and control | Provider terms; enterprise options | Full control in your environment |
| Cost model | Per token; scales with usage | Mostly fixed (GPU/server); scales with capacity |
| Low volume | Cheap | Expensive (idle hardware) |
| High steady volume | Can be expensive | Often cheaper |
| Operations | None | Deploy, monitor, upgrade, secure |
| Latency | Network + provider load | Can be very low, on your network |
| Customisation | Limited fine-tuning options | Full fine-tuning and quantisation |
The cost crossover
Self-hosting has a mostly fixed cost: the GPU server runs whether or not anyone is using it. APIs have a variable cost that grows with every token. The crossover point depends on your volume, the model you need and how well you keep the hardware busy.
A simple way to estimate: take your expected monthly tokens and multiply by the API price for a model of suitable quality. Compare that with the monthly cost of a GPU server big enough to serve your peak load with a suitable open model, plus a realistic allowance for engineering time to run it. If the API bill is far lower, use the API. If it is far higher and steady, self-hosting likely pays off. In between, a hybrid often wins.
Remember the hidden costs of self-hosting: monitoring, upgrades, security patching, capacity planning and on-call. These are real, though modest for teams already running infrastructure.
Quality
Frontier API models lead on the hardest reasoning, coding and long-context tasks. For summarisation, extraction, classification, drafting and RAG over internal documents, current open-weight models are very capable, especially when fine-tuned. Test on your own tasks; leaderboards don’t measure your use case.
Privacy
Self-hosting is the cleanest answer when contracts or regulation forbid sending data to third parties. We run this setup ourselves: Dhii Chat uses Open WebUI with models served locally via Ollama. For less strict requirements, enterprise API agreements may suffice.
Tooling
- Ollama — easiest to run; great for single servers and low concurrency.
- vLLM — high throughput with continuous batching; our usual choice for production APIs.
- llama.cpp — efficient on CPUs and smaller GPUs with quantised models.
- LiteLLM or similar gateways — one OpenAI-compatible endpoint in front of both hosted and local models.
A practical hybrid
Route sensitive data and high-volume simple tasks to a self-hosted model; send non-sensitive, complex tasks to a frontier API. With a gateway in front, the application doesn’t need to know the difference. See private and self-hosted LLMs and AI cost optimisation.