An e-commerce site adds a hundred product sheets in a few weeks. Three months later, half of these pages still do not appear in Google. Internal linking alone is not enough to ensure their visibility, and the crawl of the bots misses the deepest URLs. This is exactly the type of situation where a sitemap page makes a difference.
Sitemap and crawl budget: reducing resource waste
Crawling bots have a limited amount of time to explore each site. On a large site, they spend part of that time revisiting already known pages or endlessly following internal links. A well-structured sitemap directs the crawl towards the important URLs, instead of letting the bots discover your hierarchy randomly.
It is often observed that recent or poorly linked pages remain invisible for weeks. By explicitly listing these URLs in an XML sitemap file, we signal their existence to search engines as soon as they are published. Google Search Central specifies: the sitemap helps in the discovery of URLs, but not directly in indexing or ranking.
This is a nuance that many guides overlook. Submitting a URL in a sitemap does not guarantee that it will be crawled or indexed. On the other hand, failing to declare it means relying solely on internal linking for a bot to find it, which can take months on a low-authority site. To see how this principle translates into practice, the sitemap page of Info Manager illustrates a readable structure that facilitates this discovery.
Google Search Console and sitemap: an underutilized diagnostic tool
Many webmasters consider the sitemap as a file that is generated once, submitted to Google, and then forgotten. This overlooks its role as a technical dashboard.
When a sitemap is submitted in Google Search Console, you gain access to a report detailing the number of submitted URLs, the number of URLs actually indexed, and the reasons for non-indexation. This report allows for quick identification of concrete issues:
- Pages blocked by the robots.txt file when they should be accessible to search engines
- URLs returning a 404 error or a redirect loop, wasting crawl resources
- Pages marked “Discovered, currently not indexed,” indicating that Google knows them but does not prioritize them
Comparing the number of submitted URLs and the number of indexed URLs provides a direct measure of the site’s technical health. A significant gap indicates a problem with content quality, duplication, or technical configuration.
Frequency of sitemap updates
An outdated sitemap loses its usefulness. If content is regularly added, the lastmod tag in the XML file indicates to bots the last modification date of each page. Bots can then prioritize recently modified URLs.
Feedback varies on this point: some SEOs find that Google sometimes ignores the lastmod tag when it is not reliable. The best practice remains to update this date only when the content has actually changed.
XML sitemap and HTML sitemap: two files, two audiences
The two formats are often confused, even though they serve distinct needs. The XML sitemap is aimed at search engine bots, while the HTML sitemap is a navigation page intended for human visitors.
The XML file contains a structured list of URLs with metadata (modification date, update frequency, relative priority). It is placed at the root of the site and submitted via Google Search Console. No visitor consults it directly.
The HTML sitemap, on the other hand, takes the form of a standard web page with links organized by category. On a site with a complex structure (more than three levels deep), this page provides a shortcut to sections that are difficult to access from the main navigation.
When the HTML sitemap adds real value
On a small showcase site of five pages, an HTML sitemap adds no value. The interest arises when the architecture exceeds what the main menu can reasonably display:
- Sites with dozens of product categories and subcategories
- Editorial portals with archives accessible only via internal search
- Multilingual sites whose language versions are not all linked from the header
For multilingual sites, the XML sitemap can also include hreflang tags, which signal to search engines the relationships between the language versions of the same page. This is a rarely covered angle, but it avoids issues of duplicate content between languages.
Orphan pages and sitemap: catching what internal linking forgets
An orphan page is a URL that is not linked by any other page on the site. Without an internal link pointing to it, bots have no path to find it. The XML sitemap then becomes the only discovery channel.
Orphan pages often appear after a redesign when URLs change without all internal links being updated. They can also result from a CMS that generates dynamic pages without automatically integrating them into the navigation.
Identifying these pages involves cross-referencing the submitted sitemap with a crawl of the site. If a URL appears in the sitemap but is not found by any internal link during the crawl, it is an orphan. The sitemap does not replace a solid internal linking structure, but it acts as a safety net to ensure these pages do not completely disappear from search engines’ radar.
Correcting the problem at the source remains the priority: adding relevant internal links to these pages. The sitemap serves as a temporary solution, not a permanent substitute for a well-thought-out architecture.
The sitemap page, whether XML or HTML, is not a technical accessory to be checked off on an SEO list. It is a management tool that reveals the weaknesses of your site and accelerates the consideration of your content by search engines. A regularly updated sitemap utilized through Search Console provides more information about a site’s health than most superficial audits.