Measuring how often AI assistants mention you
Andy Bright
Director of Product and Technology
On this page
No company tells you how often its AI assistant recommends your business. Google and Bing now report some of what their own AI features do with your pages, and your analytics can show visits from AI assistants. But to know what the assistants actually say about you and your competitors, you have to ask them, many times, and record the answers.
This chapter explains how to do that properly, with nothing more than a spreadsheet. It also explains how to read the results, because answers vary so much that a single check can mislead you.
Cleotic, which publishes this guide, sells software that automates this kind of tracking. Everything in this chapter works by hand.
Three sources of information
| Source | What it tells you | What it misses |
|---|---|---|
| Reports from Google and Bing | How often your pages are shown or cited in their own AI features | Every other assistant, and anything said about you without a link |
| Your analytics and server logs | Visits from AI assistants, and which AI crawlers read your pages | Answers that mention you but lead to no click |
| Asking the assistants yourself | What each assistant actually says for your customers' questions, including about competitors | What real customers see, because their answers are personalised |
You need all three. The first two are free and quick to set up. The third takes the most work, and it's the only one that shows whether assistants recommend you.
Reports from Google and Bing
Google Search Console and Bing Webmaster Tools are free tools from Google and Microsoft that show site owners how each search engine sees their site. You need an account for each, and to prove you own your site, before you see any reports. The technical SEO guide's first chapter links to both, or your developer can set them up.
Google Search Console has a generative AI performance report that shows impressions, meaning how often a link to your site was shown in AI Overviews and AI Mode. It doesn't show clicks, or the questions people asked.
Bing Webmaster Tools has an AI Performance report covering Copilot and Bing's own AI summaries. It shows how often your pages were cited, and a sample of grounding queries, the search phrases the AI used when it found pages it went on to cite. The report is still a preview. Since June 2026 it has also shown Citation Share(opens in a new tab), your share of the citations for a grounding query. Since October 2026, a Topics Portfolio section, in beta and shown only for sites with enough data, so smaller sites may not see it, lists searches "where other sites appear and yours does not" (Bing Webmaster Blog(opens in a new tab)).
The technical SEO guide explains both reports in its section on measuring AI visibility. Bing's grounding queries show the searches behind AI answers that cite you. The AI wrote them, not your customers, but they show which topics lead to your pages, so they're a good source of ideas for your own questions. Bing's Topics Portfolio shows searches where other sites are cited and yours isn't.
Visits from AI assistants
Google Analytics
Google Analytics 4 (GA4) is the current version of Google's free analytics service. It groups visits into channels according to where they came from, such as Organic Search (unpaid search results), Email and Referral (links on other websites). Its default channels now include one called AI Assistant, for visits "from sources like ChatGPT, Gemini, Deepseek, Copilot, or Grok" (Analytics Help(opens in a new tab)).
To see it, open Reports, then Acquisition, then Traffic acquisition, and look for the AI Assistant row.
The channel doesn't include visits from Google's AI Overviews and AI Mode. Google's help page says those are excluded, and they arrive as ordinary Google search visits that GA4 can't separate.
Check the Referral row of the same report for AI websites that the AI Assistant channel doesn't catch. If you find any, you can create a custom channel group, your own set of channel rules, under Admin, then Data display, then Channel groups (Analytics Help(opens in a new tab)). Add an AI channel whose rule matches those websites, and place it above the Referral channel, because GA4 uses the first rule that matches. If someone else set up your analytics, ask them to do it.
What analytics can't show
Some AI visits arrive with no information about where they came from, for example from desktop and mobile apps, or when someone copies a link. Analytics tools often count those as Direct (Analytics Help(opens in a new tab)), the channel for visits they can't trace. Treat your AI visit numbers as the minimum, not the total.
Much of the influence of an AI answer may never produce a click. In Pew Research Center's 2025 study of Google searches in the US, people clicked a link inside Google's AI summary on just 1% of visits (Pew(opens in a new tab)). If a customer reads an AI answer that recommends Fernhill and later types Fernhill's address into a browser, analytics counts it as a direct visit. Fernhill added "An AI assistant, such as ChatGPT" to the "How did you hear about us?" question at checkout. In the first month, about one new subscriber in twenty chose it, more than its analytics showed coming from AI assistants.
Server logs
Your server logs are the record your website's server keeps of every request, including which crawler asked for which page. Some website platforms don't give you access to them, so you may need to ask your developer or host. If a page you want cited is never fetched by OAI-SearchBot or Claude-SearchBot, something may be blocking them. The technical SEO guide explains how to check.
Asking the assistants: the method
Step 1: Build a set of questions
A prompt set is the list of questions you'll ask the assistants again and again. It should be a fair sample of what your customers really ask, so build it from evidence:
- questions customers ask your sales and support staff
- searches people make on your own site
- questions from forums and from Google's "People also ask" boxes
- Bing's grounding queries
- your most important search keywords, rewritten as the questions people would type
Cover each stage of buying. Fernhill's set includes questions like these:
| Stage | Example questions |
|---|---|
| Problem | "How do I get fresher coffee at home without buying a grinder?" |
| Category | "What's a good coffee subscription in the UK?" |
| Comparison | "Coffee subscription or buying beans from the supermarket?", "Alternatives to Northgate Roasters" |
| Situation | "Coffee subscription for an office of ten people", "Coffee subscription that delivers to Ireland" |
| Brand | "Is Fernhill Coffee any good?", "How much is Fernhill's subscription?", "What is Fernhill Coffee?" |
Keep two groups separate. Unbranded questions don't mention your business, and show whether assistants recommend you to people who haven't heard of you. Branded questions name your business, and show whether assistants describe you correctly. Mixing them in one figure makes both harder to read.
Include enough questions to cover your main products and customers, with a few ways of asking each one. For a small business with one main product, 30 to 100 questions is a reasonable start. That's this guide's suggestion, not a figure from research.
Then freeze the set. If you change the questions every month, you can't compare one month with the next. When you need new questions, add them as a separate group.
Step 2: Decide which assistants, where, and how often
- Assistants: the three or four your customers use, as the chapter on each assistant explains. Record each one separately.
- Country and language: ask from, and in the language of, each country you sell in, because answers change by location. To ask from another country, use a VPN, a service that makes your connection appear to come from another country, or ask a colleague who lives there. Fernhill keeps separate groups of questions for the UK, Ireland and Germany, and asks the German ones in German.
- Web search on: for buying questions, you want the answer the assistant gives after searching. Turn on the assistant's web search option if it has one, and check that the answer lists sources.
- How often: answers vary so much that you need several runs of each question. A routine one person can keep up by hand is to ask each question three times in each assistant, once a month. For Fernhill's 40 unbranded UK questions in three assistants, that's 360 answers a month, a day or more of work by hand. Tools can automate this, or you can start with fewer questions.
Use a clean session each time, without the history, memory or personal settings of your own account, for example a private browser window where you're not signed in. OpenAI says ChatGPT can use a person's approximate location and saved memories when it searches (OpenAI(opens in a new tab)), so your own logged-in answers aren't what a new customer sees.
Step 3: Record the same things every time
| What to record | Why |
|---|---|
| Date, assistant, country and language | So you compare like with like |
| Whether your business is named | For your mention rate |
| Whether you're recommended, just listed, or mentioned with a criticism | A recommendation and a passing mention aren't the same result |
| Which competitors are named | For your share of voice |
| Which pages are cited, yours and others' | Shows which websites shape the answer |
| The searches the assistant ran, if it shows them | Shows what your pages need to match |
| What the answer says about you | To catch wrong prices, old products and other errors |
Keep a copy of every full answer. A number without the answers behind it can't be checked later.
Step 4: Work out the measures
These measures hold up when answers vary.
Mention rate, sometimes called visibility, is the share of answers that name your business. If Fernhill is named in 122 of its 360 answers to unbranded questions, its mention rate is 34%.
Share of voice is your share of all the mentions of you and the competitors you track. If those 360 answers name businesses 900 times in total, and 122 of those mentions are Fernhill, its share of voice is about 14%.
Citation rate is the share of answers that link to at least one of your pages. Track it separately from mentions, because they often don't happen together, as the first chapter explains.
Recommendation rate is the share of answers that recommend you for the person's need, not just list you.
Accuracy is the share of answers to your branded questions that describe you correctly. Check it by reading the answers, because automatic checks miss wrong prices and old product names.
Your position in a list changes so much between answers that it tells you little. Rand Fishkin of SparkToro, whose study of how much answers vary is described in the first chapter, concluded that a "ranking position in AI" is "full of baloney". He concluded that how often a brand appears, measured across many prompts, is "likely to be decent proxies for how brands actually show up in real AI answers" (SparkToro(opens in a new tab)).
Step 5: Allow for the margin of error
Because answers vary, any rate you measure is an estimate. If you asked the same questions again tomorrow, you'd get a slightly different figure, even if nothing had changed. The margin of error tells you how far the figure could move by chance.
The more answers you collect, the smaller the margin. For a mention rate of around 30%, the usual margin of error, the range the real figure falls inside 19 times out of 20, is roughly:
| Answers collected | Margin of error | A measured 30% is probably between |
|---|---|---|
| 30 | ± 16 points | 14% and 46% |
| 100 | ± 9 points | 21% and 39% |
| 300 | ± 5 points | 25% and 35% |
| 1,000 | ± 3 points | 27% and 33% |
Comparing two periods needs a wider margin, because both figures can move by chance. For two samples of the same size, a change needs to be about one and a half times the margin of either one. With 100 answers a month and rates around 30% to 40%, that means a change of more than about 13 points, so a move from 34% to 38% shows nothing. With Fernhill's 360 answers a month, the margin is about 5 points, so a change of more than about 7 points counts as real.
The formula in the note below doesn't work for rates near 0% or 100%. If you're named in none of 90 answers, a common rule of thumb is that the real rate could still be up to 3 divided by the number of answers, about 3% here.
Treat these margins as the smallest the real uncertainty could be. They assume every answer is independent, and answers to the same question asked twice tend to resemble each other. Collecting answers to more different questions narrows the range more than asking the same question many more times.
Step 6: Know what your results can't show
Say these things in any report, so nobody reads more into the figures than they hold.
- Your answers are the ones a new user with no history would see. Real customers' answers depend on their history, memory, location and settings.
- Answers from an assistant's API, the connection developers use to send questions from their own software, can differ from answers in the app people use. Windsor.ai, which sells marketing data tools, published one such test (Windsor.ai(opens in a new tab)). A software brand appeared in 59% of the answers from ChatGPT's API, and in none of the answers from the ChatGPT app for the same questions. Record which you used.
- Assistants change. A new model, or a change to how an assistant searches, can shift results quickly, as the fall in Reddit's share of ChatGPT's citations showed in the chapter on each assistant.
- Nobody knows how often each question is asked. Some tools sell estimates of "prompt volume". Ahrefs, which sells such estimates itself, says that "with the exception of Microsoft Copilot", no major AI company shares this data, and that some vendors' figures "can be significantly over-estimated" (Ahrefs(opens in a new tab)). Use estimates to compare topics with each other, not to forecast visits.
- No tool sees inside the assistants. Google says "No third-party tool has access to our internal ranking or AI systems" (Google(opens in a new tab)). That applies to every AI tracking tool, including Cleotic's. Tools record answers from the outside, as you would by hand.
Linking what you did to what changed
When you publish a page, earn some coverage or fix a technical problem, write down the date and which questions it should affect. After a few weeks, compare the answers to those questions before and after, using the same assistants, countries and method. Then check whether the change is bigger than the margin for a change, as described above.
Even then, be careful about cause. The assistants change too, and so do your competitors. Write "our mention rate rose after we published the office page", not "rose because we published it", unless you have stronger evidence that one caused the other.
Setting goals and reporting
A useful goal names the questions, the assistants and a date. Fernhill's first goal is to be named in 55% of answers to its 40 unbranded UK questions across ChatGPT, Google's AI Mode and Perplexity within six months, up from 34% at the start. Its second is to have no wrong delivery or price information in answers to its branded questions.
When you report results:
- Give each figure with its range, for each assistant. For example, "Named in 22% to 38% of ChatGPT's answers, up from 7% to 19% in April." With fewer answers per assistant, each range is wider than the overall one.
- Show two or three real answers, so people can see what the numbers mean.
- Report mentions, recommendations, citations and accuracy separately.
- Add visits from AI assistants and the "How did you hear about us?" figures, noting that both undercount.
- List what you did, and what you'll do next.
- Say what you don't know, such as how much of a change came from the assistants themselves.
How often to check
| How often | What to look at |
|---|---|
| Weekly | Visits from AI assistants, and any badly wrong answers about your business |
| Monthly | Your prompt tracking results, Search Console's generative AI report, Bing's AI Performance report and grounding queries, and AI crawler activity in your logs |
| Every three months | Your goals, a new group of questions if needed, and the AI companies' documentation for changes |
What to do next
- Find the AI Assistant row in GA4's Traffic acquisition report, and note last month's figure.
- Add "An AI assistant, such as ChatGPT" to your "How did you hear about us?" question.
- Look at Search Console's generative AI report and Bing's AI Performance report, and copy twenty grounding queries into a list.
- Write 30 questions across the stages of buying, split into branded and unbranded.
- Ask each question three times in each of three assistants, record the answers in a spreadsheet, and work out your mention rate with its margin of error.
Further reading
- AIs are highly inconsistent when recommending brands or products(opens in a new tab), SparkToro's study and its advice on measuring.
- Default channel group(opens in a new tab), Google's definition of GA4's channels, including AI Assistant.
- Introducing Topics Portfolio in AI Performance(opens in a new tab), Bing's October 2026 update.
- How Ahrefs estimates AI search demand(opens in a new tab), on what prompt volume figures can and can't tell you.