How to track whether AI answers name your brand

Ask an AI assistant who the best agencies are and you will get a list. Ask again and you will get a different one. Measuring that honestly takes more than one question.

Written by the Colleviate Strategy desk

Why a single answer tells you nothing

SparkToro, working with Gumshoe.ai, tested this. As reported by Search Engine Journal, the team ran 12 prompts, such as which chef's knife or headphones to buy, between 60 and 100 times each on ChatGPT, Claude and Google's AI. The same list of brands almost never came back twice, and the order was effectively random. The brands that led a category did keep turning up across runs, which is the useful part: the signal is how often you appear, not where.

We are quoting a secondary report of that study, not its raw data, and your categories will differ. The method that follows does not depend on the exact numbers. It depends on the one point the reports agree on: one answer is one draw from a distribution.

Why it is worth measuring at all

When an answer is read and not clicked, being named is the only visibility left. Pew Research Center found that people clicked a traditional search result in 8 percent of visits that showed an AI summary, against 15 percent without one, and clicked links inside the summary in 1 percent. That is the argument for measuring citation, and for measuring it as carefully as share of search.

Step 1: write the question set

Twenty to thirty questions, in the words a buyer would use, in four groups:

• Category: “best marketing agency for a B2B brand in Mumbai”.
• Comparison: “agency retainer or project fee”.
• Problem: “why has our paid media stopped working”.
• Brand: “what does your brand do”, to catch errors.

Take the wording from real conversations (sales calls, enquiry emails, the questions customers actually asked), as customer research on a small budget shows, not from your keyword list.

Step 2: run each question many times

Run every question at least 10 times per engine per month, in fresh sessions, logged out where the tool allows, with the same location and settings each month. For each run record: whether your brand is named, whether a page of yours is cited, whether what was said about you is right, and which competitors appeared. Note the date and the model name shown.

Step 3: read it as a rate, with its margin

Your mention rate is the runs that named you divided by the runs. That number has a margin of error, and it is wide at small samples. For a true rate of 30 percent, 10 runs give a reading that can swing by roughly 28 points either way, 40 runs by about 14, and 100 runs by about 9. So a move from 30 to 40 percent on 10 runs is noise. Only report a change when it is larger than the margin for your sample, and run more for the questions that matter most.

Runs per question	Margin on a 30% reading, 95% confidence
10	about 28 points
40	about 14 points
100	about 9 points

Step 4: report it next to what you already track

Put the mention rate beside your share of search and your share of voice, against the same named competitors, as in share of search vs share of voice. If you appear more often in AI answers than your search share suggests, the content is working. If less, start with why AI answers do not mention your brand. Review it monthly, as part of a reporting cadence the board already reads, and in long-consideration categories such as the ones in automotive marketing in India look at it over quarters, not weeks.

If you buy a tool

Ask the vendor how many times each prompt is run per engine and how the margin is shown. A dashboard that reports a single position for your brand in an AI answer is reporting a number the research above treats as noise.