Technical SEO in 2026
Andy Bright
Director of Product and Technology
On this page
Technical SEO is the work of making sure search engines and AI assistants can find, read and store the pages on your website. It's different from writing good content or getting other sites to link to you. Those things decide whether your pages are worth showing. Technical SEO decides whether search engines and AI assistants can see your pages in the first place.
This guide is for marketers, founders and small-business owners who look after a website but aren't developers. It assumes no technical knowledge. Each term is explained when it first appears, and each chapter says what you can check yourself and what to ask a developer to do.
It covers Google and Bing, and the AI assistants that people increasingly use to search, such as ChatGPT, Claude, Perplexity, Microsoft Copilot and Google's own AI Overviews. Every claim links to the source it comes from, usually the search engine's or AI company's own documentation.
Start here
If you're new to this, read How search engines and AI assistants find your pages first. It follows one page from a small online shop through every step, from being published to appearing in a Google result or an AI answer. Every other chapter builds on it. If you meet a term you don't know, the glossary explains it in plain English.
The chapters
- How search engines and AI assistants find your pages. The whole process from start to finish, and the words used to describe it.
- Crawlability, robots.txt and AI crawlers. How to control which crawlers can visit your site, which AI companies send crawlers, and whether to let them use your content for AI training.
- Indexing, canonicals and duplicate content. How to keep pages out of search results when you need to, and how search engines choose between two addresses for the same page.
- Site architecture and internal linking. How the links between your pages help crawlers find them, and how to spot pages that nothing links to.
- Redirects and site migrations. How to move pages or change your domain without losing your search traffic.
- XML sitemaps and IndexNow. How to give search engines a list of your pages, and tell Bing straight away when something changes.
- JavaScript rendering. Why content that appears through JavaScript can be invisible to AI crawlers, and how to check your pages.
- Page experience and Core Web Vitals. How Google measures page speed, and how much it affects your rankings.
- Structured data. The code that can add star ratings, prices and other details to your search results, and what it does for AI answers.
- International SEO and hreflang. How to set up a site that has versions for different countries or languages.
- Technical SEO for AI search. The settings each AI company gives you, the file called llms.txt, and how to tell whether AI assistants are citing you.
- Audit checklist. Every check from every chapter in one list, in priority order, starting with ten you can do without a developer.
- Glossary. Every term used in the guide, explained in plain English.
If you only have time for a few, read chapters 1, 2 and 7, then go through the checklist in chapter 12.
What AI search changes
For Google, appearing in AI answers depends on the same things as appearing in its ordinary results. Google says a page has to be "indexed and eligible to be shown in Google Search with a snippet" to appear as a link in AI Overviews or AI Mode, and that there are "no additional technical requirements" (Google(opens in a new tab)). That means the page has to be stored in Google's index and allowed to show a short description in results.
The other AI assistants add three complications. Each AI company runs its own crawlers, which your site has to let in. Most of those crawlers don't run JavaScript, according to the best public study (Vercel and MERJ, 2024(opens in a new tab)), so they can miss content that appears through it. And many websites now block AI crawlers through their hosting or security settings, often without anyone deciding to. Chapter 1 explains each of these, and chapter 2 shows how to check your own site.
What changed in 2025 and 2026
This table is mainly for readers who already know the basics and want to catch up. Each change is explained in the chapter linked from it.
| When | What happened | What to do about it |
|---|---|---|
| Jul 2025 | Cloudflare started asking(opens in a new tab) the owner of every new domain whether to allow AI crawlers. Over a million customers had already switched on its AI crawler block. | Check your CDN settings (chapter 2). |
| Sep 2025 | Cloudflare introduced Content Signals(opens in a new tab), a way to state in robots.txt whether your content can be used for search, AI answers and AI training. | Optional. Add the line if you want to state a preference (chapter 2). |
| Dec 2025 | Google updated its JavaScript rules(opens in a new tab). A noindex tag changed by JavaScript "is not well defined", and pages that don't return a 200 status may not be rendered. | Put indexing instructions in the page's HTML, not in JavaScript (chapter 7). |
| Feb 2026 | Google documented(opens in a new tab) that Googlebot reads only the first 2 MB of each page. Bing launched AI Performance reports(opens in a new tab) showing when Copilot cites your pages. | Set up Bing Webmaster Tools if you haven't (chapter 11). |
| Mar 2026 | Google moved the files(opens in a new tab) that list its crawlers' IP addresses. | Ask your developer to check any firewall rules that use them (chapter 2). |
| May 2026 | Google stopped showing FAQ rich results(opens in a new tab) on 7 May, and published its guide to optimising for AI features(opens in a new tab) on 15 May. | Don't add FAQ markup to win rich results (chapter 9). |
| Jun 2026 | Google announced a Search Console setting(opens in a new tab) to opt out of its AI features, and a report showing AI impressions. It also said Google Search ignores llms.txt. Bing added Citation Share(opens in a new tab). | Look at the new reports (chapter 11). |
| Aug 2026 | The Search Console AI setting(opens in a new tab) and report reached every site on 31 August. Version 2 of llms.txt(opens in a new tab) was published. | Check the setting on each Search Console property you own. |
| Sep 2026 | Google said page experience factors other than Core Web Vitals "don't directly help your website rank higher"(opens in a new tab). | Fix pages that fail Core Web Vitals, and don't chase perfect scores (chapter 8). |
| Oct 2026 | Google's Chrome team added experimental speed measurements for single-page apps(opens in a new tab). Bing added a Topics Portfolio(opens in a new tab) to its AI reports. | If your site is a single-page app, ask your developer to review the new data (chapter 8). |
Questions without a settled answer
Some questions in this area don't have an agreed answer yet. The guide sets out the evidence on each, and says what it recommends.
Whether to let AI companies train their models on your content depends on what your content is worth to you. There are good reasons on both sides, set out in chapter 2. The guide's recommendation is to treat training and AI search as two separate decisions, because blocking one doesn't block the other.
llms.txt is a proposed file that summarises a website for AI tools. Google Search says it ignores the file, while Google's own website-testing tool, Lighthouse, checks for it. Google's John Mueller has called it "purely speculative for now". Chapter 11 recommends it only for sites with developer documentation.
Since August 2026, Google lets you opt your whole site out of its AI features. Doing so removes all the visits and impressions those features would send you. The guide recommends staying in unless you have a licensing reason to leave.
Whether structured data helps with AI answers is disputed. Google says no special markup is needed for its AI features. Some SEO practitioners argue AI systems use it more than Google says. Chapter 9 recommends it for other reasons anyway.
Whether AI crawlers run JavaScript is uncertain. The best public study, from December 2024(opens in a new tab), found the major AI crawlers don't, and no AI company has documented it either way. The guide assumes they don't.
What this guide can't promise
No technical change guarantees a ranking, a citation in an AI answer or a recommendation. Search engines and AI assistants make their own choices, and AI answers change from one person, place and day to the next. Google's guidance on SEO tools(opens in a new tab) says outside tools "don't have access to our internal ranking data. They can't guarantee performance." The same is true of any guide or consultant.
What technical SEO can do is remove the problems that stop search engines and AI assistants using your pages at all.
Further reading
Search engines change their documentation often. These two pages list Google's changes as they happen:
- Google Search documentation updates(opens in a new tab), for changes to how Google crawls, indexes and shows pages.
- Google crawling documentation changelog(opens in a new tab), for changes to Google's crawlers.
These are good next steps from the search engines themselves:
- In-depth guide to how Google Search works(opens in a new tab), Google's own explanation of crawling, indexing and showing results.
- AI features and your website(opens in a new tab), Google's short page on how AI Overviews and AI Mode choose pages.
- Optimizing your website for generative AI features on Google Search(opens in a new tab), Google's longer guide, including the tactics it says you can skip.