Free comparison page Work with us

How We Test AI Visibility

By Mehmet, Top Seed AI · Published · Updated

In short: we write buyer questions without your brand name, ask each in a fresh chat, record who is named and which sources are cited, and repeat the same questions across assistants and over time. A one-off check is a single reading. Improvement is judged only by rerunning the same questions.

This is how we run an AI visibility check, so you know what the numbers in a report mean and what they do not.

1. We write buyer questions

We start from the questions real customers ask about your category and products, for example "best lightweight tents for a first backpacking trip" or "best project management software for a small agency". The questions do not contain your brand name, because a shopper who does not know you cannot ask for you by name. A one-off check uses a small set of these. Ongoing tracking covers up to 50.

2. We ask about you by name

We add a small number of questions that name your brand, such as "What is [brand] and what do they make?". This shows whether the assistant knows you and describes you correctly. We report these answers apart from the buyer questions, because in our checks a brand usually looks better when you ask about it by name than when you ask a buyer question.

3. We use a fresh chat for every question

Each question runs in a new session with no history (a new search, for Google AI Overviews), so earlier answers do not influence later ones. We keep the wording identical every time we repeat a check.

4. We record what the assistant says

5. We repeat across assistants and over time

A one-off check, like the one behind our free comparison page, is a single reading: we ask an AI assistant with web search 10 questions your buyers ask. For ongoing work, we load your question set, up to 50 questions, into a monitoring tool that runs it every week across five AI surfaces: ChatGPT, Perplexity, Gemini, Google AI Mode and Google AI Overviews. We record what changes week to week. Same questions, same wording, so you can compare as closely as possible, though not perfectly. The weekly runs are automated, and automated runs may differ from what a person sees in a normal chat. Model updates, location and logged-in state can also change answers, and we cannot hold those constant.

How we count

Our main number is the mention rate: the share of buyer-question runs, per assistant, in which your brand is named. We look at patterns across questions and across repeated runs, not at the position of a single answer. The reason is that answers vary. In a study by SparkToro and Gumshoe (600 volunteers, 12 prompts, nearly 3,000 runs), the odds of getting the same list of brands twice were under 1 in 100, and the authors advise tracking how often a brand appears across runs instead of its exact position. That is also why we treat a one-off check as a single reading that shows where to look, and use the repeated weekly runs as the basis for comparison.

How we measure improvement

When we work together, we agree in writing, before we start, which questions we will re-test and what counts as improvement. We run the same questions at the start and again at the end of your first month, using repeated runs rather than a single one, and report what changed and what did not, including when nothing did. We do not call a shift real unless it is larger than the run-to-run variation we measured across those repeated runs. If you continue after the first month, we keep re-running the same questions every week and send a short report every month.

Our own test of 57 brands

The "30 of 57" figure on our home page comes from our own test. We picked 57 small and mid-size consumer brands, mostly in sports and outdoor, and asked four or five buyer questions about each, mostly in ChatGPT. Thirty of the brands were not named in any of the buyer questions we asked. We chose the brands and wrote the questions ourselves, and ran the test in early October 2026, so it is a single test, not a random sample, and it does not describe all brands. Each answer names only a handful of brands, so many brands will be absent by design. Read it as "being absent is common among the brands we tried", not as a rate for the market. Results vary by assistant and by day. It is our own test, not an independent study.

What the results can and cannot tell you

Questions people ask

Why do you leave my brand name out of the questions?

Because a shopper who does not know you cannot ask for you by name. Questions without your name show whether you are named to people who are still choosing.

Why a fresh chat for every question?

So earlier answers and chat history do not influence later ones. We keep the wording identical when we repeat a check.

Is the "30 of 57" figure a benchmark for my industry?

No. We chose the brands ourselves, mostly in sports and outdoor, so it is not a random sample and does not describe all brands.

How do you know whether something improved?

We agree in writing, before work starts, which questions we will re-test and what counts as improvement, then run the same questions again, with repeated runs, and report what changed and what did not. We do not call a shift real unless it is larger than the run-to-run variation we measured.

Sources

External figures were checked against the linked pages on Oct 15, 2026. Where a study comes from a company that sells related tools, we say so. None of the outside studies measures your brand.

About this guide. Written by Mehmet, who runs Top Seed AI. We sell AI visibility measurement and improvement, so we have an interest in this topic; the sources above are there so you can check our reading. Updated Oct 15, 2026: rewritten with sourced figures, an FAQ and clearer limits.

Want a head start?

We show you which pages AI reads in your field, then write the comparison page you're missing, ready to publish.

Get your free comparison page

Work with us