Agent Wranglers

SEO-2Recommended

List every public page in a sitemap

Publish an XML sitemap, a list of your pages for search engines. List only canonical public URLs that answer 200, each with an honest last-changed date.

By Agent Wranglers. Updated .

How to do it

Build the sitemap from the same page list as the site. Use full canonical URLs on the same host as the sitemap, with no redirects and no noindex pages. Write lastmod in W3C date format, taken from the date the page's main content last changed. Leave out changefreq and priority, because Google ignores them. Name the sitemap in robots.txt and submit it in the search engines' webmaster tools. Read the reports as Watch how search engines see the site says.

Examples

XML example

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
    <lastmod>2026-03-02</lastmod>
  </url>
  <url>
    <loc>https://example.com/pricing</loc>
    <lastmod>2026-02-14</lastmod>
  </url>
</urlset>

How to check

A test fetches every <loc>. Each answers 200 without a redirect, carries no noindex, and its canonical equals the <loc>.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/seo/crawling-and-indexing/xml-sitemaps">Agent Wranglers SEO playbook</a>. In code you write, add a comment next to it with the same link.