AI traffic
AI traffic shows the AI crawlers that request your pages and the visitors AI assistants send to your site. It needs your site connected first; see Connect your site.
Tracking starts after installation and does not backfill earlier visits. A crawler visit doesn't confirm indexing, training use, or a recommendation in an AI answer, and a referral count is not a count of unique people or sales.
Read the traffic view
The summary shows AI referrals, answer fetches, crawler requests, and AI share when eligible total pageviews are available. The crawler table lists each crawler's purpose and the share of its requests that were verified. Use the date range, domain, provider, crawler, source, and page-search filters to narrow the evidence, and Crawlers only or Referrals only to separate the two.
The page table shows visit counts, systems involved, latest activity, and trends. Open a row to inspect recent events for that page.
Pages that bots crawl at least 20 times in 30 days but that get two or fewer AI referrals appear in Opportunities as traffic gaps, with suggested changes for the page.
AI crawlers
Cleotic counts requests from documented AI crawlers and groups them by why the crawler fetched the page.
| Purpose | What it means | Examples |
|---|---|---|
| Answer fetch | An AI assistant fetched the page while answering someone's question. This is the closest crawler signal to real demand. | ChatGPT-User, Claude-User, Perplexity-User, meta-externalfetcher, Google-Agent |
| Search index | A crawler built an AI search index. Being indexed makes a page eligible to appear in answers. | OAI-SearchBot, Claude-SearchBot, PerplexityBot, meta-webindexer |
| Training | A crawler collected content to train AI models. Training doesn't make a page appear in answers. | GPTBot, ClaudeBot, Amazonbot, meta-externalagent, CCBot |
Answer fetches in the summary counts answer fetches only. Crawler requests counts all three purposes. CCBot belongs to Common Crawl, whose public dataset many AI models train on, rather than to Anthropic. See Common Crawl's crawler documentation.
A crawl doesn't confirm that a page was cited, and training crawls don't lead to referrals. Check whether important product and comparison pages receive answer fetches and search index crawls. If a page is missing, inspect its public accessibility, internal links and crawl rules with your site team or Site readiness.
What isn't counted
- Link previews, such as facebookexternalhit, Twitterbot and Slackbot. They fetch a page when someone shares a link.
- Search engines, such as Googlebot, Bingbot and Applebot. Their indexes also feed Google AI Overviews, AI Mode and Copilot, but the same requests serve ordinary search.
- Robots.txt control names. Google-Extended and Applebot-Extended control how Google and Apple use your content but never send requests of their own, so a request that uses either name is fake. See Google's crawler documentation.
- AI-sounding names that no operator documents, such as DeepSeekBot and PanguBot. Anyone can send them.
- AI browsers on a person's own computer, such as Perplexity Comet and Claude in Chrome. They look like ordinary browsers.
Verified requests
A crawler names itself in its user agent, and anyone can copy that name. For each crawler request, Cleotic checks the IP address against the list of crawler addresses the operator publishes. Requests from outside the list are spoofed and aren't counted.
The crawler table shows the share of each crawler's requests that were verified:
- Verified requests came from an address on the operator's list. Meta publishes no list, so its crawlers are checked against Meta's own network.
- No IP list means the operator publishes no addresses, as with Bytespider and YouBot. Their requests are counted but can't be checked.
- A request is also unverified when the connection doesn't send the visitor's IP address. The WordPress plugin doesn't send it, so WordPress crawler requests are counted as unverified.
AI referrals
AI referrals are visits Cleotic recognises as arriving from an AI assistant. Recognised sources include ChatGPT, Claude, Perplexity, Gemini, Copilot, Meta AI, You.com, Le Chat, Grok, Character.AI, and DuckDuckGo AI.
Server-side integrations inspect the referring URL and recognised utm_source values. Browser tracking detects a recognised AI referrer through its beacon. WordPress also offers an optional frontend beacon for referrals on cached pages.
Referral data does not identify the exact answer or prompt behind a visit. Check which pages receive visits and whether the pattern persists after a content change.
AI share
AI share needs a total-pageview denominator as well as AI events. The browser snippet and WordPress's AI events only mode record AI evidence, not every pageview, so their share is incomplete. Missing total-pageview data is not 0% AI traffic.
When visits are missing
- Confirm the site is connected and active.
- Widen the date range and clear crawler, source or page filters.
- Check that the domain and subdomain match the installation.
- For browser tracking, remember that a crawler that only fetches HTML won't run the snippet.
- For server tracking, check delivery and whether caching or request filtering excludes the traffic.
Robots.txt rules may stop a cooperative crawler from requesting a page, and a firewall may block a request before it reaches your origin. Some assistants and browsers remove referral information, so a visit with no recognised referrer or campaign source may not be attributed to AI. A connected site with no referral rows can still be working correctly.
Related
Updated