OpenAI-compatible API: what it means and what it covers

An OpenAI-compatible API accepts the same requests as OpenAI's Chat Completions endpoint and answers in the same shape, so code written with an OpenAI SDK runs against it after changing the base URL, the API key and the model name. It is a promise about the format of the conversation, not about the models, the features or the other endpoints behind it.

So compatibility is a spectrum, not a checkbox. This post covers what the term reliably includes, where providers part ways, how to test one yourself, and where P/9 stops.

Compatible means one endpoint, not the whole OpenAI API

When a provider calls itself OpenAI-compatible, it almost always means POST /v1/chat/completions: a model string and a messages array in, a choices array with the assistant's message and a finish_reason out, plus a usage block counting prompt, completion and total tokens. OpenAI's SDKs let you replace the base URL (the Python SDK also reads OPENAI_BASE_URL), so anything answering in that shape can sit behind them.

OpenAI's own API is much larger: embeddings, images, audio, files, fine-tuning, and the Responses API, which OpenAI's Python SDK now calls the primary way to use its models, with Chat Completions supported indefinitely. A provider can match chat completions exactly, offer none of the rest, and still be accurately called compatible.

Open-source servers such as vLLM, Ollama and llama.cpp expose that endpoint too, so that existing OpenAI clients can connect to them. Ollama's documentation says plainly that it supports a subset of the OpenAI API. The useful question is never whether a provider is compatible, but with which endpoints and which fields.

The parts that almost always match

The core of a chat request is what providers get right, because every client exercises it on the first call: system, user and assistant messages in order; temperature, top_p, stop and max_tokens; and the reply text at choices[0].message.content.

Streaming is usually faithful too: server-sent events, one chat.completion.chunk per data line carrying a delta, ending with data: [DONE]. Token counts come with one caveat. On a stream, OpenAI sends usage only when the request sets stream_options with include_usage, so confirm a provider does the same before building cost tracking on it.

If your application sends messages and reads text, it will very likely run unchanged anywhere that uses the word.

Tool calling and JSON output depend on the model server, not only the request

In OpenAI's shape, you send a tools array of functions with JSON Schema parameters; the model answers with tool_calls whose arguments are a JSON-encoded string; you return each result as a message with the role tool and the matching tool_call_id. A provider can accept every one of those fields and still differ in whether the model calls tools reliably, calls several in parallel, or supports each tool_choice mode.

Much of that is fixed when the model server starts. vLLM, a widely used open-source serving engine, accepts tool_choice set to auto only when launched with --enable-auto-tool-choice and a tool-call parser for the model. Two providers serving the same open weights can answer the identical request differently.

Structured output splits the same way. OpenAI's JSON mode (response_format of type json_object) produces valid JSON; Structured Outputs (type json_schema) also guarantees it matches your schema. A provider may support both, one or neither. Where the field is quietly ignored, a prompt that asks for JSON can still produce something JSON-like most of the time, which fails later and less clearly than an error would.

The differences that do not throw an error are the expensive ones

Model ids rarely carry over, and models.list() is missing on some compatible APIs, so the model string is always a line you change. Errors match in outline more than in detail: OpenAI's envelope carries message, type, param and code, and its Python SDK raises a named exception for 400, 401, 403, 404, 409, 422, 429 and 5xx. OpenAI reports an exhausted credit balance as a 429; a gateway that answers 402 for the same condition raises the SDK's generic status error instead, and your retry logic may not expect it.

Rate-limit headers such as x-ratelimit-remaining-requests may be sent, renamed or absent, leaving the 429 as the only signal. Hardest to see is a parameter accepted and ignored, because it returns 200. Anthropic's OpenAI SDK compatibility page is candid about this: it lists response_format, seed and logprobs as ignored, requires n to be exactly 1, and says most unsupported fields are silently ignored rather than producing errors. Where a provider publishes no such table, only testing tells you.

How do you test a provider's compatibility yourself?

A hello-world request proves the base URL and the key, and nothing else. Test the requests your application actually sends, against the model you intend to use, and start with tools, where the serving configuration shows through most directly: send the same request to each candidate and compare the replies.

• Tools: the reply has tool_calls, every arguments value parses as JSON, and finish_reason is tool_calls
• Structured output: send your real schema, then a prompt that pulls the model away from it, and validate the result
• Streaming: a long answer ends with a finish_reason and data: [DONE], and usage arrives if you track cost from the stream
• Parameters: values whose effect is visible, such as n of 2 or a very low max_tokens, are reflected in the response
• Errors: a bad key, an unknown model, a spent budget and a burst past the rate limit each produce the status and exception your code handles
• Endpoints: every other one you call, models.list() and embeddings included, actually exists

curl "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "the-model-you-will-use",
    "tool_choice": "auto",
    "tools": [{ "type": "function", "function": {
      "name": "get_order_status",
      "parameters": { "type": "object",
        "properties": { "order_id": { "type": "string" } } }
    }}],
    "messages": [{ "role": "user", "content": "Where is order 4417?" }]
  }'

The alternatives share the shape, so choose on what the shape cannot express

Most OpenAI API alternatives now speak this shape: model labs with compatibility layers, such as Anthropic and Google; providers serving open-weight models, such as Together, Fireworks and DeepInfra; gateways such as OpenRouter; and your own vLLM server. Because the shape is shared, moving a workload between them is a base URL, a key and a model string, plus whatever your tests turned up.

That pushes the real decision onto what the API does not model at all: which models you need, which country executes the request, what is kept afterwards, and what currency you are billed in. With no residency requirement, a large global provider is a sensible answer. The picture changes when where a request ran is itself the requirement.

What a sovereign AI gateway is, and when location becomes the requirement

Where P/9 is compatible, and where it stops

P/9 serves POST /v1/chat/completions, synchronous and streamed, with messages, stream, max_tokens, temperature, usage and finish_reason in the OpenAI shape, streams ending in data: [DONE], and failures in an OpenAI-shaped error envelope carrying a message and a type. Its compatibility stops there.

Model ids are P/9 route ids from the catalogue, such as paranine/gpt-oss-120b(Global), and GET /v1/models answers 404. Embeddings, images, audio and files are not part of the documented API, there is no fine-tuning endpoint, and a client built on the Responses API gets a 404 until it is pointed at chat completions. Errors carry P/9's own types (auth_error, model_scope_error, proxy_error), and an exhausted organisation budget or key credit limit answers 402 rather than 429.

Fields the gateway does not read itself, tools and response_format among them, pass through to the model server behind the route, so tool support is a property of the route rather than of P/9 as a whole: test the one you plan to use. If you depend on the Responses API, fine-tuning or OpenAI's own models, use OpenAI directly. P/9 fits when chat completions is your shape and billing in rupees, per-key limits and a named pool behind every route matter.

Moving an OpenAI-compatible codebase to P/9: base URL, route id, key scope and cost

The short version

OpenAI-compatible means an OpenAI client can talk to the provider, not that the provider does everything OpenAI does. Messages, streaming and token counts almost always match; tools, structured output, other endpoints, errors and ignored parameters are where providers differ.

Test what you actually send before you move anything, then choose on what the API cannot express. If you need something only OpenAI offers, use OpenAI.