What should an XML sitemap contain?
Search engines use automated crawlers to discover pages and gather content for processing. Discovery and indexing are separate steps and are not guaranteed to happen immediately. Large or complex websites may need particular attention to ensure important pages can be found.
What is an XML sitemap?
An XML sitemap lists URLs you want search engines to discover and consider for indexing. It can help crawlers:
- Discover the listed URL;
- Identify the preferred URLs submitted by the site
- Notice significant new or updated content
It can also help surface pages without internal links , often called orphan pages. However, a sitemap does not replace useful internal links: fix the missing connections where appropriate.
Alongside URLs, the protocol supports fields such as last modification dates, suggested change frequency and relative priority. Some extensions can describe language alternatives. Support differs by search engine: Google ignores changefreq and priority. The file is mainly for machines rather than visitor navigation.
When is an XML sitemap particularly useful?
A sitemap is particularly useful for:
- Large websites with many pages or substantial archives
- New sites with few backlinks or limited discovery paths
- Online shops with many products and complex URL systems, provided the sitemap contains intended canonical pages rather than every filter variant
- Sites that publish or substantially update content frequently
- Sites where some important pages are difficult to discover through navigation
The structure of an XML sitemap
The file uses a machine-readable format with the following components:
Core elements
- XML declaration: identifies the XML version and may declare the character encoding.
- urlset: contains the URL entries and declares the relevant sitemap namespace.
- url: represents an individual entry. Each requires a loc element containing the full URL, including its scheme, such as HTTPS.
Optional fields and extensions
- changefreq: a suggested change frequency defined by the protocol. Google ignores it.
- lastmod: the date of the page's last significant modification. It should reflect actual content changes.
- priority: a relative priority value from 0.0 to 1.0 within the site. Google ignores this field; it is not a ranking boost.
- Language alternatives: an appropriate hreflang extension can list equivalent versions for different languages or regions.
Accurate supported fields can provide useful information, but extra tags do not override crawling or indexing decisions. Use only information you can maintain reliably.
XML and HTML sitemaps: the difference
An XML sitemap supplies URLs and supported information to search engines. An HTML sitemap is a visitor-facing page of organised links. Both can help discovery, but neither guarantees indexing or tells Google to rank a page in a particular way.
Practical guidance for creating and maintaining a sitemap
- Automate generation where appropriate. A sitemap can be maintained manually, but content management system, plugins or a suitable build process can reduce manual work. Check the generated output rather than assuming automation makes it correct.
- Use canonical versions. Avoid listing unnecessary duplicate URLs or parameter variants when another address is the preferred version.
- List the pages you genuinely want indexed. They should be accessible, useful and consistent with the site's other canonical and indexing signals.
- Exclude noindex pages to avoid submitting conflicting intentions.
- Make the sitemap available and submit it through suitable webmaster tools or reference it with a Sitemap directive in robots.txt.
- Maintain lastmod accurately. Do not reset it for every build or cosmetic edit when the main content has not changed significantly.
- Split files when needed. A sitemap is limited to 50,000 URLs and 50 MB uncompressed. Use multiple files and a sitemap index when the limits require it.
- Keep canonical signals consistent, especially where filters, sorting or tracking parameters create alternative addresses.
- Include appropriate final URLs. Do not list error responses, login-only pages or pages blocked from the intended crawler.
- Maintain clear internal navigation as well. A sitemap supports discovery but does not replace a useful network of links.
A short summary
An XML sitemap gives search engines a maintained list of preferred URLs. It is particularly useful for larger, growing or frequently updated websites. A clean file supports discovery and consistent signals, but does not guarantee faster indexing or higher visibility.
Review the existing pages, implement reliable generation and record significant changes accurately in lastmod. Check the public file and its submitted status when the website changes.