Why we run every prompt three times

Ask ChatGPT the same question twice and you will often get two different answers, built from different searches, citing different websites. The engines are probabilistic by design. So a single test of "does AI mention my business" is close to worthless: it can show you present when you usually are absent, or absent when you usually are present. In our audits we run every prompt three times on every engine and report how often you appear, because a rate is something you can act on and retest. This is the n=3 protocol, and the reasoning behind it is just arithmetic.

The same question, different answers

Three things vary between runs of an identical prompt. The wording of the answer changes, which matters little. The set of searches the engine runs during fan-out changes, which matters a lot, because a different search pool means different pages get read. And the final citations change, which is the part your customers see. In our audit work it is completely normal to watch a brand get named in one run and skipped in the next, with nothing about the brand or its website having changed in between.

The arithmetic of one test

Suppose the truth is that an assistant names you in one out of every three answers about your service. You do not know that yet. You open the app and ask once.

Both outcomes feel like evidence. Neither is. And the failure compounds when you test a change: post a new page, ask once, see your name, and you will credit the page for what may be ordinary variance. Agencies demo this trick live on sales calls, sometimes without knowing it is a trick.

Why three, and not one or ten

Three runs per prompt per engine is a floor, chosen deliberately. One run cannot distinguish luck from pattern at all. Ten runs would be more precise, and for a monthly retainer we do rerun and track over time, but for a diagnosis the marginal precision is not worth tripling the cost. Three runs is the smallest number that separates the outcomes a business owner actually needs to act on:

• 0 of 3. You are effectively invisible for this question. The work starts at finding out whether you were even retrieved.
• 1 or 2 of 3. You are in the rotation. The engine considers you a plausible answer and does not consider you the answer. This is usually the cheapest ground to gain, because the retrieval problem is already solved.
• 3 of 3. You own this question today. The job is defense: keep the page current and watch who is climbing.

The same rule applies to reading anyone else's data, including ours. When a report says "AI recommends your competitor," the first question to ask is: out of how many runs? A finding that does not come with a denominator was a screenshot, and screenshots are how this industry sells fear.

What this looks like in a report

Every finding in an EZ Web audit carries its rate: named in 2 of 3 runs on ChatGPT, 0 of 3 on Perplexity, and so on, per question, per engine. When we later measure whether our work moved anything, the comparison is rate against rate, run the same way. It is slower and roughly three times more expensive for us than screenshotting one lucky answer. It is also the difference between measurement and theater.