Does my AI provider store my prompts?

Most do, for a limited window, usually for abuse review and debugging, and the window commonly differs by product tier and by whether a zero-retention agreement is in place. The only reliable answer is the one your provider gives in writing for your account, because the public documentation often describes a different tier than the one you are on.

Does P/9 store prompts and completions?

No. The gateway retains no prompt and no completion on any route. A request log row carries the model, the key, the project, the status, the timings, the token counts and the settled cost, and nothing of what was said.

Is a no-training promise the same as no retention?

No, and conflating them is the most common mistake in a vendor review. A provider can truthfully say your data never trains a model while keeping prompts for thirty days, which is still a copy of your data on infrastructure you do not control.

How can I tell what a provider retains?

Ask four questions in writing: is content stored at all after the response, for how long, who inside the company can read it, and does any of that change by product tier or plan. Vague answers to specific questions are themselves an answer.

Can a provider debug a failure without keeping prompts?

Yes, on metadata, which is what P/9 does. Latency, time to first token, throughput, token counts, status codes and error classes diagnose almost every real production problem. What metadata cannot do is let a provider read back a bad answer, which is a genuine operational cost and the reason zero retention is a real commitment rather than a marketing line.

All posts

8 Sep 20267 min readparanine

Prompt logging: what your AI provider keeps after the answer

Most inference providers retain prompts and completions for a period. What gets kept, how to find out before your data is in it, and what P/9 records instead.

Most inference providers keep your prompts and the model's replies for some period after the response, usually for abuse review and debugging, and the length commonly depends on your product tier and on whether you signed a zero-retention agreement. That is the honest general answer, and it is why the specific one has to come from your provider in writing.

P/9 keeps neither, on any route. A request log row records the model, the key, the project, the status, the timings, the token counts and the settled cost, and nothing of what was said.

Retention is not one setting, it is four questions

A single yes or no about logging hides the parts that decide a vendor review. Ask these separately, because a provider can answer one of them reassuringly while the others carry the risk.

Is the content stored at all once the response has been returned?
If so, for how long, and in which country does that copy sit?
Who can read it: an automated abuse classifier, a support engineer on a ticket, or both?
Does any of the above change by product tier, plan or contract, and which one are you actually on?

Training and retention are different promises

This is the distinction that gets lost, and it is usually lost in the customer's favour by accident and against them in fact. A commitment not to train on your data says nothing about whether a copy exists. Both statements can be entirely true at once, and only one of them is about whether your customer's message is sitting on somebody else's disk.

For a regulated buyer, retention is the harder question, because a retained copy in another jurisdiction is reachable by that jurisdiction regardless of the contract governing it. A no-training promise does not survive that; it was never about that.

Why a prompt is a data transfer under Pakistan's localisation rules

Why keeping nothing is harder than it sounds

Retention exists for real reasons, not lazy ones. Abuse review is easier with the text. A support engineer debugging a bad generation would rather read the exchange than infer it. Replaying a failed request against a fixed model is the fastest way to confirm a fix.

Giving those up costs the provider something, which is precisely why the commitment means something when it is made. It also forces the platform to be diagnosable on metadata alone, and that constraint tends to produce better instrumentation than a provider who can always fall back on reading the prompt.

What a log has to keep to still be useful

Almost every production question about an inference workload is answerable without the content. Which route was slow, which key spent the budget, which project caused the spike, what fraction of calls failed and with which error class, how many tokens a workload actually uses against what was estimated.

That is what P/9 records per request, and it is enough to run capacity, cost and incident work on. What it cannot do is tell you what a user asked, which is the point.

The model and route that served the call, and the pool behind it
The key and project it was billed to
Status, latency, time to first token and throughput
Input and output token counts
The settled cost in rupees, the same figure the ledger debits

Turning those token counts into a rupee figure

How to check any provider in ten minutes

Send the four questions above to your account contact and ask for the answer in writing, naming your plan. Then read the answer for two specific things: whether it distinguishes training from retention, and whether it gives a duration or a posture.

A provider that answers with durations, tiers and named roles has thought about this and can be held to it later. A provider that answers with a commitment to taking privacy seriously has told you what it can be held to, which is nothing.

The short version

Assume content is retained unless a provider says otherwise for your specific tier, and treat a no-training promise as answering a different question than the one you asked. The strongest position is a provider whose logs are complete enough to run the platform and empty of anything your customer said.

Common questions

Does my AI provider store my prompts?: Most do, for a limited window, usually for abuse review and debugging, and the window commonly differs by product tier and by whether a zero-retention agreement is in place. The only reliable answer is the one your provider gives in writing for your account, because the public documentation often describes a different tier than the one you are on.
Does P/9 store prompts and completions?: No. The gateway retains no prompt and no completion on any route. A request log row carries the model, the key, the project, the status, the timings, the token counts and the settled cost, and nothing of what was said.
Is a no-training promise the same as no retention?: No, and conflating them is the most common mistake in a vendor review. A provider can truthfully say your data never trains a model while keeping prompts for thirty days, which is still a copy of your data on infrastructure you do not control.
How can I tell what a provider retains?: Ask four questions in writing: is content stored at all after the response, for how long, who inside the company can read it, and does any of that change by product tier or plan. Vague answers to specific questions are themselves an answer.
Can a provider debug a failure without keeping prompts?: Yes, on metadata, which is what P/9 does. Latency, time to first token, throughput, token counts, status codes and error classes diagnose almost every real production problem. What metadata cannot do is let a provider read back a bad answer, which is a genuine operational cost and the reason zero retention is a real commitment rather than a marketing line.