XML sitemaps and IndexNow

Updated 16 October 2026 · 10 min read

Andy Bright

Director of Product and Technology

On this page

What an XML sitemap is
Do you need a sitemap?
What belongs in a sitemap
Getting the dates right
Submitting your sitemap
IndexNow
Checking that it works
What to do next
Further reading

Crawlers find most pages by following links, which can take time. A sitemap and a service called IndexNow let you tell search engines about your pages directly instead of waiting to be found.

Neither one forces a search engine to do anything. Google calls a sitemap "merely a hint"⁠(opens in a new tab), and Bing says IndexNow "does not guarantee that web pages will be crawled or indexed"⁠(opens in a new tab). They do help search engines find new and changed pages sooner. That matters more now that AI assistants answer questions from search indexes. If Fernhill Coffee changes its subscription prices, it wants AI answers to quote the new prices, not last month's.

This chapter explains:

what an XML sitemap is, and whether you need one
what belongs in a sitemap, and what to leave out
why the dates in a sitemap matter, and how they go wrong
how to submit a sitemap to Google and Bing
what IndexNow is, and how to set it up

What an XML sitemap is

An XML sitemap is a file on your website that lists the pages you want search engines to know about. XML is a text format for storing information in a structured way, using labels in angle brackets. Next to each page's address, the sitemap usually gives the date the page last changed.

Here is a short sitemap for Fernhill Coffee, the example shop in this guide, which lives at www.example.com:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.example.com/subscriptions</loc>
    <lastmod>2026-10-16T09:00:00+01:00</lastmod>
  </url>
  <url>
    <loc>https://www.example.com/products/ethiopia-yirgacheffe</loc>
    <lastmod>2026-10-14T15:30:00+01:00</lastmod>
  </url>
</urlset>

Each <url> block describes one page. The <loc> line is the page's full address, and the <lastmod> line is the date and time the page was last changed.

Most website platforms create a sitemap automatically. It usually lives at an address like /sitemap.xml or /sitemap_index.xml. You rarely need to write one by hand, but you should check what's in yours.

Do you need a sitemap?

Not always. Google says a site might not need one⁠(opens in a new tab) if it has around 500 pages or fewer that you want in search results, if those pages are well linked to each other, and if it doesn't have lots of video, images or news articles that you want found. A sitemap helps most when:

the site is large, so crawlers might miss new or updated pages
the site is new, and few other websites link to it yet
you publish a lot of images, video or news
your pages change often
some pages aren't well linked, such as free tools or campaign landing pages

Because almost every website platform creates a sitemap for free, most sites have one anyway. The useful question is whether yours is accurate.

What belongs in a sitemap

A sitemap should list only the pages you want to appear in search results (Google's sitemap guide⁠(opens in a new tab)). That means each page's main address, and only pages that search engines are allowed to index. Leave out:

addresses that redirect to another page
addresses that return an error, such as a 404 or a server error
pages with a noindex tag
duplicate versions of a page, such as addresses with tracking codes, print versions or filtered shop pages
addresses that robots.txt blocks
test pages, preview pages and your site's own search results

Google treats a page's presence in your sitemap as a weak signal⁠(opens in a new tab) about which address is the main version of that page. If your sitemap lists the wrong version, it contradicts your canonical tags. The indexing chapter explains canonical tags and duplicate pages.

Rules for the file

These rules come from Google⁠(opens in a new tab) and Bing⁠(opens in a new tab).

One sitemap file can hold up to 50,000 addresses, or 50 MB before compression. Larger sites split their addresses across several sitemaps and list those files in a sitemap index, a sitemap of sitemaps. Bing says an index can list up to 50,000 sitemaps.

Write full addresses, starting with https://, exactly as you want them crawled. Save the file in UTF-8, the standard text encoding.

Keep sitemaps at the root of your site. Unless you submit a sitemap through Google Search Console, it only covers pages in its own folder and below. A sitemap at /blog/sitemap.xml can't list a page at /products/.

The order of the addresses in a sitemap doesn't matter to Google.

There are also special kinds of sitemap for images, video and news, and sitemaps can list the language versions of each page. The international SEO chapter covers language versions.

Getting the dates right

A sitemap can include three optional details for each page: <lastmod>, <priority> and <changefreq>. Google ignores <priority> and <changefreq> completely.

<lastmod> is the one that counts, but Google only uses it if it's "consistently and verifiably" accurate (Google's sitemap guide⁠(opens in a new tab)). It should be the date of the last real change to the page, meaning a change to its main text, its structured data or its links. Google's own example of a change that doesn't count is updating the copyright year in the footer.

Bing takes the same view. Its July 2025 article on sitemaps⁠(opens in a new tab) says Bing ignores changefreq and priority, that lastmod "remains a key signal", and that it should include the time as well as the date, in the standard ISO 8601 format shown in Fernhill's example above.

Dates usually go wrong in one of two ways.

The first is a sitemap that gives every page the same date: the date the site was last rebuilt. Some website tools do this automatically. Every page then appears to change every day, so the date tells search engines nothing, and they learn to ignore it on your site.

The second is changing dates on purpose without changing the pages, to make the site look fresh. The SEO consultant Lily Ray lists artificial content refreshing⁠(opens in a new tab) among the tactics that can backfire. Change the date when the page changes, and only then.

Keep the date consistent everywhere it appears. The sitemap's lastmod, the dateModified value in the page's structured data and any "last updated" date shown on the page should all describe the same change. The structured data chapter explains dateModified.

Submitting your sitemap

There are three ways to tell Google about a sitemap⁠(opens in a new tab). Most sites should use the first two.

Submit it in Google Search Console, in the Sitemaps report, and in Bing Webmaster Tools. Both are free services where you can see how search engines treat your site.
Add a line to your robots.txt file. Every crawler that reads robots.txt, including AI crawlers, can then find your sitemap:

Sitemap: https://www.example.com/sitemap.xml

Submit it through the Search Console API, which is useful if you manage a lot of websites.

Bing says it processes sitemaps regularly, and recommends using them alongside IndexNow⁠(opens in a new tab), which is explained below.

Keeping a sitemap accurate

A sitemap stays accurate when it's created automatically from the same information as your pages. New pages then appear in it as soon as they're published, and deleted pages disappear.

Larger sites benefit from splitting the sitemap by section, such as /sitemap-shop.xml, /sitemap-blog.xml and /sitemap-guides.xml, and submitting each one separately. Search Console then shows how many pages in each section are indexed, which makes problems much easier to find.

Make sure every public page you want found is included, especially pages that your menus and blog don't link to, such as free tools and landing pages.

Sitemaps on Next.js websites

This section is for developers. Websites built with the Next.js framework can generate a sitemap from a file called app/sitemap.ts (Next.js documentation⁠(opens in a new tab)). The official examples set lastModified: new Date(). That's fine for a demonstration, but on a real site it stamps every page with the time of the last build, which is the first date problem described above. Take the date from your content system instead:

// app/sitemap.ts
import type { MetadataRoute } from 'next'
import { getPublishedArticles } from '@/lib/cms'

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  const articles = await getPublishedArticles() // canonical, indexable only

  return articles.map((article) => ({
    url: `https://www.example.com/guides/${article.slug}`,
    lastModified: article.contentUpdatedAt, // last real content change, not build time
  }))
}

Check which date your content system provides. Many update their "published" or "updated" date every time someone saves or republishes a page, even when nothing a reader would notice has changed. If yours does, keep a separate field for the last real change.

For large sites, generateSitemaps splits the output into several files. The Next.js example includes a comment noting that "Google's limit is 50,000 URLs per sitemap". Next.js sitemaps can also list language versions through the alternates.languages property.

IndexNow

IndexNow is a free, open service for telling search engines that a page has been added, changed or deleted. You send the page's address to one participating search engine, and the notification is "automatically shared with all other participating search engines"⁠(opens in a new tab).

As of October 2026, IndexNow's list of participants⁠(opens in a new tab) is Bing, Yandex, Seznam, Naver, Yep, the Internet Archive and Amazonbot. Google doesn't take part. For Google, your sitemap and your links are still how new pages get found, so IndexNow adds to a good sitemap rather than replacing it.

IndexNow matters for AI search mainly because of Bing. Microsoft Copilot answers questions using Bing's index. When Bing launched its AI Performance report⁠(opens in a new tab) in February 2026, it recommended IndexNow so that AI systems can "reference the most current version of a page".

Check whether you already have it

You may not need to set anything up. Bing lists website platforms with IndexNow built in⁠(opens in a new tab), including Wix Studio, Duda, GoDaddy and Shopify. It also lists WordPress plugins that can switch it on in a few minutes, including Yoast, Rank Math, AIOSEO and SEOPress. Some CDNs offer it as a setting too. Bing Webmaster Tools shows whether your pages are already arriving through IndexNow.

Setting it up yourself

If your platform doesn't support IndexNow, it's a small job for a developer. The steps come from the IndexNow documentation⁠(opens in a new tab).

The first step is to create a key, a code that proves the notifications come from you. It can be 8 to 128 characters long, using letters, numbers and dashes.

The second step is to put the key on your site, as a text file at the root of your domain, named after the key. For example, https://www.example.com/0f8b6c4e2a9d4b7e.txt, containing just the key. You can put the file somewhere else on the same site and give its location in each notification, but the location limits which pages you can submit, so the root is the recommended place.

The third step is to send the addresses of new or changed pages. A single address can be sent with a simple web request:

curl "https://api.indexnow.org/indexnow?url=https://www.example.com/subscriptions&key=0f8b6c4e2a9d4b7e"

Up to 10,000 addresses can be sent together in one request:

curl -X POST "https://api.indexnow.org/indexnow" \
  -H "Content-Type: application/json; charset=utf-8" \
  -d '{
    "host": "www.example.com",
    "key": "0f8b6c4e2a9d4b7e",
    "keyLocation": "https://www.example.com/0f8b6c4e2a9d4b7e.txt",
    "urlList": [
      "https://www.example.com/subscriptions",
      "https://www.example.com/products/ethiopia-yirgacheffe"
    ]
  }'

The response comes back with a status code:

Code	What it means
200	The addresses were received
202	The addresses were received, and the key is still being checked
400	The request wasn't in the right format
403	The key wasn't valid, because the file wasn't found or doesn't match
422	The addresses don't belong to your site, or the key doesn't match the required format
429	Too many requests, which IndexNow treats as possible spam

A 200 only means the notification arrived. It doesn't mean the page has been crawled or indexed.

When to send a notification

Send one every time you publish, update or delete a page. Deleted pages are worth sending too, so search engines drop the old address sooner.

Don't send your whole site's history. Bing's guidance is to submit only the pages that have changed since you started using IndexNow⁠(opens in a new tab).

Don't send the same unchanged pages over and over. Bing confirms that "every crawl counts towards your crawl quota"⁠(opens in a new tab), including crawls that IndexNow triggers.

The simplest setup uses a webhook, a message your website platform sends automatically when something happens, such as a page being published. The webhook triggers a short piece of code that sends the page's address to IndexNow, while the sitemap updates its lastmod date at the same time:

// Call from your website platform's publish webhook
export async function notifyIndexNow(urls: string[]) {
  const res = await fetch('https://api.indexnow.org/indexnow', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json; charset=utf-8' },
    body: JSON.stringify({
      host: 'www.example.com',
      key: process.env.INDEXNOW_KEY,
      keyLocation: `https://www.example.com/${process.env.INDEXNOW_KEY}.txt`,
      urlList: urls,
    }),
  })
  if (![200, 202].includes(res.status)) {
    console.error('IndexNow submission failed', res.status)
  }
}

Checking that it works

Google Search Console's Sitemaps report lists each sitemap you've submitted, when Google last read it, and any errors it found.

Search Console's Page indexing report can be filtered to show only the pages in one sitemap. You can then compare the pages you submitted with the pages Google indexed, and see why the others weren't.

Bing Webmaster Tools shows the status of your sitemaps and a history of your IndexNow submissions.

A developer can also run a regular check of their own. This script fetches every address in a sitemap and lists any that don't return a 200, which catches most problems before search engines find them:

# List sitemap addresses that don't return 200
curl -s https://www.example.com/sitemap.xml \
  | grep -oE '<loc>[^<]+' | sed 's/<loc>//' \
  | while read -r u; do
      c=$(curl -s -o /dev/null -w "%{http_code}" "$u")
      [ "$c" != "200" ] && echo "$c $u"
    done

What to do next

Open your sitemap and check that your most important pages are listed.
Check that it doesn't list redirects, noindex pages or duplicate addresses.
Look at the lastmod dates. If every page has the same date, ask your developer to use each page's real last-changed date.
Add a Sitemap: line to your robots.txt, and submit the sitemap in Google Search Console and Bing Webmaster Tools.
Find out whether your platform supports IndexNow, and switch it on if it does.