What an LLM gateway does, and what changes when it runs in Pakistan

An LLM gateway is a single API in front of many language models. Your application sends one request shape to one base URL, and the gateway resolves the model id to a backend, checks the key, enforces whatever limits that key carries, records what the call cost, and returns the answer.

An LLM gateway running in Pakistan adds two things a foreign endpoint structurally cannot, whatever the contract says: an invoice in the currency the business earns in, and an answer to the question of which country executed the request. This post covers both, starting with the ordinary engineering case, because a gateway earns its place before either of those matters.

Why put anything between your application and the model

Without a gateway, a provider key ends up in every service that calls a model, a model id ends up hardcoded across a dozen repositories, and the only view of what any of it costs arrives at the end of the month as a single number. None of that is wrong on day one. All of it becomes expensive at the same moment, which is the first time something has to change.

The failures are predictable enough to list. A model is deprecated and every service naming it needs a deploy. A key leaks, and revoking it takes down whatever else was quietly using it. A batch job loops and nobody notices until the balance is gone. Finance asks which team spent this, and the honest answer is that the data was never collected.

The four jobs a gateway actually does

Routing is the one people name first and the one that matters least. The other three are why the pattern survives contact with production.

• Resolution: one id per route, so changing what serves that route is an operational change rather than a code change in every caller
• Identity and scope: a key that is capped in rupees, limited to a rate, restricted to specific models and expiring on a date, enforced at the edge rather than by convention
• Accounting: the cost of each request attributed to a key and a project as it happens, instead of reconstructed from an invoice afterwards
• Observability: latency, time to first token, throughput and error class per route, in one place, across every model you call

Every team builds this once, and then stops

The first version is a small proxy that forwards a request and adds a header. It stays small until the first incident, and then it grows a key store, a rate limiter, a spend ledger, a retry policy, a fallback route and a metrics pipeline, all of which have to be correct while under load and none of which are the product.

That is the real argument for a gateway as a product rather than a file in your repository. Not that the proxy is hard to write, but that the parts added after the first outage are the parts you cannot afford to get wrong.

How per-key rate limits, spend caps and expiry work on P/9

What changes when the gateway runs inside the country

Two things change that no amount of contractual language reproduces from abroad. The first is the invoice. A rupee-denominated bill needs no international card, no foreign balance held against a forecast nobody trusts, and no exchange-rate line to explain to a finance function that approved a different number.

The second is jurisdiction. A server answers to the law of the country it sits in, regardless of who rents it or what their policy says. For a bank, a hospital, a telco or a government department that is usually the question deciding whether a project ships, and it is the one an engineering team is least likely to have asked, because nothing in the API surfaced it.

Latency is the third thing people expect to matter and it usually matters least. A request that does not cross an ocean saves a fixed cost on every call, which is real but small against the time a model spends generating a long answer. Treat it as a bonus rather than the argument.

What a sovereign AI gateway is, and how to tell a real one from a badge

Two pools behind one API

P/9 runs two kinds of route through the same interface. The PK NPU-native pool is served on Huawei Ascend hardware P/9 operates inside Pakistan. The global pass-through routes reach models served elsewhere, and are marked as such in the catalogue.

The point is not that everything runs locally. It is that the pool which served a call is a property you choose in the model id and can read back afterwards, rather than a detail the gateway abstracts away. A provider that cannot answer that per request is not offering residency, whatever the landing page says.

The catalogue: every route, its pool, and its rate per million tokens in rupees

What adopting one costs you

Two values. The gateway serves the OpenAI chat completions and embeddings shapes, so any OpenAI SDK works once the base URL and the key change, and the model id names the route you want.

curl https://api.paranine.com/v1/chat/completions \
  -H "Authorization: Bearer $P9_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "paranine/gpt-oss-120b(Global)",
    "messages": [{ "role": "user", "content": "One line on what a gateway is for." }]
  }'

Moving an OpenAI-compatible codebase across, in one change

The short version

A gateway earns its place for reasons that have nothing to do with where it runs: one place for keys, limits, cost and routing. Running it in Pakistan adds the two things a foreign endpoint cannot offer at any price, which are a bill in rupees and a jurisdiction you can name.

If neither is a requirement for you, a global aggregator with a larger catalogue is a reasonable answer and an honest recommendation. If one of them is, it is usually not negotiable.