Skip to content
dhii.ai

Comparison

Self-Hosted LLM vs API — Cost, Privacy and Performance

Should you self-host an open-weight LLM with vLLM or Ollama, or use a hosted API? A practical comparison of cost at different volumes, privacy, quality, latency and operational effort.

Updated

Short answer: hosted APIs win on top-end quality, zero operations and low or spiky volumes. Self-hosting wins when data must not leave your environment, when volumes are high and steady, or when a smaller open model is good enough for the task. Many organisations end up hybrid.

Comparison

Hosted API Self-hosted open model
Quality ceiling Highest (frontier models) Very good; close for many tasks
Privacy and control Provider terms; enterprise options Full control in your environment
Cost model Per token; scales with usage Mostly fixed (GPU/server); scales with capacity
Low volume Cheap Expensive (idle hardware)
High steady volume Can be expensive Often cheaper
Operations None Deploy, monitor, upgrade, secure
Latency Network + provider load Can be very low, on your network
Customisation Limited fine-tuning options Full fine-tuning and quantisation

The cost crossover

Self-hosting has a mostly fixed cost: the GPU server runs whether or not anyone is using it. APIs have a variable cost that grows with every token. The crossover point depends on your volume, the model you need and how well you keep the hardware busy.

A simple way to estimate: take your expected monthly tokens and multiply by the API price for a model of suitable quality. Compare that with the monthly cost of a GPU server big enough to serve your peak load with a suitable open model, plus a realistic allowance for engineering time to run it. If the API bill is far lower, use the API. If it is far higher and steady, self-hosting likely pays off. In between, a hybrid often wins.

Remember the hidden costs of self-hosting: monitoring, upgrades, security patching, capacity planning and on-call. These are real, though modest for teams already running infrastructure.

Quality

Frontier API models lead on the hardest reasoning, coding and long-context tasks. For summarisation, extraction, classification, drafting and RAG over internal documents, current open-weight models are very capable, especially when fine-tuned. Test on your own tasks; leaderboards don’t measure your use case.

Privacy

Self-hosting is the cleanest answer when contracts or regulation forbid sending data to third parties. We run this setup ourselves: Dhii Chat uses Open WebUI with models served locally via Ollama. For less strict requirements, enterprise API agreements may suffice.

Tooling

  • Ollama — easiest to run; great for single servers and low concurrency.
  • vLLM — high throughput with continuous batching; our usual choice for production APIs.
  • llama.cpp — efficient on CPUs and smaller GPUs with quantised models.
  • LiteLLM or similar gateways — one OpenAI-compatible endpoint in front of both hosted and local models.

A practical hybrid

Route sensitive data and high-volume simple tasks to a self-hosted model; send non-sensitive, complex tasks to a frontier API. With a gateway in front, the application doesn’t need to know the difference. See private and self-hosted LLMs and AI cost optimisation.

FAQ

Frequently asked questions

Is self-hosting always more private?

It gives you the most control, but only if the server itself is secured well. Enterprise API agreements with no-training terms and regional processing can also meet many privacy requirements.

Can we switch between the two?

Yes, if your application calls models through an OpenAI-compatible gateway. Then moving workloads between hosted APIs and self-hosted models is mostly configuration.

Related services

More guides

Start a project

Have a workflow that should run itself?

Tell us what you want automated or built. You get a written assessment of feasibility, risks and a fixed quote — usually within two business days.