An XML sitemap is a list of the URLs you want search engines to know about. It does not make pages rank and it does not force them into the index, but it helps crawlers discover new and deep pages sooner, and it gives you a clean set of URLs to compare against what Google has actually indexed.
A sitemap is only useful if search engines trust it. A file full of redirects, noindexed pages and fake modification dates teaches crawlers to ignore it. This guide covers what belongs in a sitemap, the size limits, how lastmod is really used, and how to submit the file and read the reports that come back.
Only canonical, indexable URLs that return 200
Every URL in the sitemap is a statement: "this is a page I want in search results". Treat it like one. Include a URL only if all of these are true.
- It returns a 200 status. Leave out 301 and 302 redirects (list the destination instead), 404s and 410s.
- It is the canonical version. If page A has a canonical tag pointing to page B, list B only. Pick one protocol and host (https, with or without www) and use it consistently.
- It is indexable: no noindex robots tag or X-Robots-Tag header.
- It is not blocked by robots.txt. A sitemap URL that robots.txt disallows produces a warning in Search Console.
- It is a real page, not a URL with tracking parameters, session IDs or a sort order.
Size limits and sitemap index files
A single sitemap file can hold at most 50,000 URLs and must be no larger than 50 MB uncompressed. You can gzip it to save bandwidth, but the 50 MB limit applies to the uncompressed size. Sites larger than that split their URLs across several sitemaps and list those files in a sitemap index, which can itself reference up to 50,000 sitemaps.
Splitting is useful even below the limit. Separate sitemaps for products, categories, articles and static pages let you see indexing coverage per section in Search Console, so a problem in one template stands out instead of being averaged away.
Place sitemaps at the root of the host. Under the sitemaps protocol, a sitemap at /blog/sitemap.xml should only list URLs under /blog/. All URLs must use the same host as the sitemap unless you have verified ownership of both hosts in Search Console.
<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
<loc>https://www.example.com/sitemaps/products.xml</loc>
<lastmod>2026-11-14</lastmod>
</sitemap>
<sitemap>
<loc>https://www.example.com/sitemaps/articles.xml</loc>
<lastmod>2026-11-15</lastmod>
</sitemap>
</sitemapindex>lastmod matters; priority and changefreq do not
Each url entry can carry three optional tags. Google has said it ignores priority and changefreq. It does use lastmod, but only when the dates are consistently and verifiably accurate. A lastmod should change when the main content of the page changes: the text, a price, the structured data. It should not change because a footer year updated or a related-posts widget reshuffled.
The common mistake is a generator that stamps every URL with the build date. When Google sees all 5,000 pages "modified" every day and finds nothing new on them, it stops trusting lastmod for that site, and the one signal that could speed up recrawling of genuinely updated pages is lost. If you cannot produce real modification dates, leave lastmod out.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://www.example.com/guides/fix-common-json-errors</loc>
<lastmod>2026-11-03</lastmod>
</url>
<url>
<loc>https://www.example.com/pricing?plan=team&billing=annual</loc>
</url>
</urlset>Encoding and format details
Sitemaps are XML, so the usual XML rules apply. The file must be UTF-8 encoded, and five characters must be escaped inside a loc value: & as &, < as <, > as >, and quotes as " and '. An unescaped ampersand in a query string is the single most common reason a sitemap fails to parse. URLs with spaces or non-ASCII characters should be percent-encoded.
Extensions exist for images, videos, news articles and hreflang alternates. They are worth adding only when they carry information the page itself does not make easy to find, such as video files loaded by a player or language alternates on a large international site.
Telling search engines where the sitemap is
There are two reliable ways. First, add a Sitemap line with the full URL to robots.txt; every major crawler reads it, and it can appear anywhere in the file. Second, submit the sitemap in Google Search Console under Sitemaps and in Bing Webmaster Tools. Google retired its old sitemap ping endpoint in 2023, so scripts that "ping" Google after each deploy no longer do anything; an accurate lastmod is the replacement.
After submission, the Sitemaps report in Search Console shows whether Google could fetch and parse the file and how many URLs it discovered. "Couldn’t fetch" usually means the URL is wrong, blocked, or returning an error to Googlebot. For the more useful view, open the Pages report and filter it by sitemap: it shows how many of the URLs you submitted are indexed and why the rest are not. A large group of "Crawled - currently not indexed" pages from one sitemap is a sign that section has a quality or duplication problem, not a sitemap problem.
# robots.txt
User-agent: *
Disallow: /admin/
Sitemap: https://www.example.com/sitemap_index.xmlA workflow for small and static sites
Most CMS platforms and frameworks generate sitemaps automatically, and that is the best option when it is available because the file stays in sync with the site. For a static site, a set of landing pages or a migration, the Neemtools Sitemap Generator turns a pasted list of URLs into a valid sitemap.xml, removing blank lines and duplicates and escaping special characters. Untick the lastmod option unless every page really changed today.
Then generate or update robots.txt with the Robots.txt Generator, which adds the Sitemap line for you, upload both files to the site root, and submit the sitemap in Search Console. Revisit the Pages report a week or two later to see how the submitted URLs are being indexed.