Self-hosting an LLM vs an API: which is actually cheaper?

Self-hosting an LLM is cheaper than paying an API per token only when the hardware stays busy, at high and steady utilisation for most hours of the month. Below that point the API is cheaper, because it charges for the tokens you send, while a server you own costs the same whether it is busy or idle.

The honest comparison is therefore a method rather than a price: the monthly cost of hardware, power and people, set against the tokens your traffic will really put through them. Every input below is a variable you fill in, because hardware prices, tariffs and throughput depend on choices only you can make.

An API charges for tokens, and a server charges for time

A per-token API bills in proportion to use: send twice the traffic and the bill doubles, send nothing and it stops. Owned hardware is the opposite. The purchase, the rack and the engineers who keep it running are a fixed monthly cost, and traffic only changes how many tokens that cost is spread across.

A server becomes a cost per token only once you know how many tokens it will serve. On P/9 the per-token side is published per million tokens in rupees, with input and output priced separately, and an idle account accrues no charge. Turn those rates into a monthly bill for your workload before comparing anything.

What a million tokens actually costs in rupees, and how to price a workload

The break-even is a formula you fill in, not a number anyone can quote you

Write self-hosting as one fixed monthly figure and the API as the bill for the same traffic. Each line is something you can get a real quote or a real measurement for, and none of it should come from a blog post, this one included.

We leave every input blank on purpose. A break-even filled in with our numbers would describe somebody else's company, and a throughput figure given without the model, the batch shape, the sequence lengths and the serving stack is not a measurement.

F = H + E + S + O     fixed monthly cost of self-hosting

  H   hardware: purchase price / months of useful life, or monthly rent
  E   energy: average kW x 730 hours x tariff per kWh, plus cooling
  S   people: share of engineering time x loaded monthly cost
  O   everything else: rack space, network, spares, monitoring

A                     the API bill for the same traffic, per month

Self-hosting is cheaper when F < A.
(730 is the hours in an average month: 8,760 in a year / 12)

Utilisation is the variable that decides the answer

F is fixed; the tokens it buys are not. If C is what the hardware can serve in a month at a latency you accept, and u is the share of that your traffic actually fills, each token costs F divided by the tokens actually served, which is C times u. At half the utilisation every token costs twice as much, and at a quarter, four times as much. Nothing else in the formula moves the answer that fast.

Set that cost equal to the API's rate and you have the break-even utilisation. If it comes out above 1, the hardware cannot beat the API for this workload at any volume. If it comes out at 0.7, self-hosting wins only if the hardware is at least 70 percent busy on average across the month, nights and weekends included.

Self-hosted cost per million tokens = F x 1,000,000 / (C x u)
Break-even utilisation              u* = F x 1,000,000 / (C x p)

  C   tokens per month the hardware can serve at acceptable latency
  u   share of that capacity your traffic fills, from 0 to 1
  p   the API's cost per million tokens for your workload:
      A x 1,000,000 / your tokens per month

You size for the busiest hour and pay for it all night

Hardware has to be sized for the busiest hour, or requests queue exactly when users are waiting, and it is then paid for through every quiet hour. Averaged over a month, utilisation can therefore be no higher than your average load divided by your peak load. An application used during one country's working day idles through most of the week; a pipeline running steadily around the clock is where self-hosting looks its best.

Redundancy lowers the ceiling again. If one machine can carry the peak, surviving a hardware failure takes a second, and two machines where either could carry everything can average at most half busy.

The costs that never appear on the hardware quote

The quote covers the machine. What follows recurs, and it is why S and O are rarely as small as the first estimate. The item most often forgotten is the layer in front of the model: a model server answers requests, but it does not know who sent them, what they may spend or which team to charge.

• On-call: a server that fails at night fails for your users, and someone has to be reachable whether or not anything breaks
• Model swaps: newer weights mean conversion, re-testing and re-measuring, and a larger model needs memory of roughly its parameter count times the bytes per parameter, so an upgrade can become a purchase
• Patching: drivers, runtimes and the serving framework all ship updates, and each is a change to a production system
• Toolchain: if the accelerators you can buy are Huawei Ascend rather than Nvidia, add model conversion and operator work on CANN rather than CUDA
• Currency: hardware priced in dollars can change its rupee cost between quote and payment
• Access and metering: keys, per-key limits, spend attribution and request logs, none of which a raw model server provides

What an LLM gateway does, and why every team ends up building one

When self-hosting is the better choice

Sometimes it plainly is. Only the first case below is a cost argument. The other three are requirements, and a cheaper per-token rate does not overturn a requirement.

• Sustained, steady utilisation above the break-even, such as a batch pipeline that runs continuously
• An air gap: if no request may leave your own network, no API meets that, local or foreign, P/9 included
• Weights or serving changes your provider does not offer, such as a fine-tuned model of your own; P/9 has no fine-tuning endpoint
• Hardware you already own: if it is paid for and idle, H is a sunk cost, and the comparison is power and people against the API bill

Reserved capacity is a third meter between the two

Renting hardware by the hour for your organisation alone replaces the purchase with an hourly price but leaves utilisation where it was, because a reservation is still a cost for time. P/9 offers this as dedicated inference: Ascend NPUs reserved for one organisation, billed per hour of held capacity and quoted per organisation. Its own page states the rule plainly: reserved hardware you do not keep busy is the most expensive way to buy inference.

Dedicated Ascend NPU capacity, billed per hour of held time

Measure your traffic before you buy anything

Run the workload on a per-token API first, where idle costs nothing, and read the real inputs off it: tokens per month, the split between input and output, and how traffic spreads across the week. Then measure C on that same traffic shape. Serving stacks reach their throughput by batching many concurrent requests, so a figure taken under heavy load overstates what the hardware delivers at your real concurrency.

The move is not a one-way door. P/9 serves the OpenAI chat completions shape, and so do common self-hosted stacks: vLLM, for one, ships an OpenAI-compatible server. Keep the base URL and the model id in configuration, and moving a workload in either direction is a deploy rather than a rewrite.

The short version

A self-hosted LLM is a fixed monthly cost looking for enough tokens to spread across, and utilisation decides whether it finds them: halve it and every token costs double. If the hardware would be busy around the clock, or an air gap, custom weights or hardware you already own decides for you, run your own. Otherwise per-token pricing is the cheaper meter until your own measurements say otherwise.