ShipworkSite inspection
Sitemap diff

Crawlability

PaginationAre paginated pages set up right?Internal searchIs site search eating crawl budget?URL parametersAre parameters creating duplicates?Log analyserWhat does Googlebot actually crawl?Robots builderNeed a robots.txt file?Soft 404sAre pages “not found” but returning 200?JavaScriptCan crawlers see it without running JS?FreshnessIs my site quietly going stale?Robots.txtDoes robots.txt say what I think?

Indexing

Sitemap lastmodAre my sitemap dates valid?SitemapIs my sitemap actually fetchable?IndexingIs Google allowed to index my pages?CanonicalsIs Google indexing the wrong URL?Index signalsDo sitemap and index signals agree?Sitemap generatorNeed a sitemap.xml for my site?Bulk URL statusWhere does each of these URLs really land?Redirect builderNeed the redirect rules?RedirectsAre my URLs answering directly, over HTTPS?

On-page

Keyword ideasWhat are people searching for?SERP previewHow does my page look in Google?Content qualityAre your pages too thin or too alike?Image weightAre images slowing the page?CannibalizationAre pages competing with each other?On-page checkDoes the page use its target phrase?ImagesAre my images accessible and loading?Meta tag builderWhat should my title and share tags say?DuplicatesDo my pages compete for one query?

Links

Orphan pagesWhich pages can no link reach?Link graphHow deep do your pages sit?Anchor textDo links say what they point at?Outbound linksAre external links still alive?Broken linksAre internal links sending visitors nowhere?

Structured data

Rich resultsIs my markup eligible for a rich result?Structured dataIs my product schema valid?Schema coverageDo my key pages carry structured data?Social previewHow does my page look when shared?Schema vs pageDoes markup match the page price?Schema builderNeed valid JSON-LD?

International

Hreflang sitemapDo page and sitemap hreflang agree?HreflangDo my language versions link back?Hreflang builderNeed the hreflang tags?Page DoctorWhy is this page not doing well?Redirect planMy site has dead links — what redirects do I write?Sitemap diffIs anything missing from my sitemap?Robots simulatorWhat does my robots.txt actually block?Can Googlebot?Can Googlebot fetch this URL?
Every error, explainedGuides
Pricing Learn Error guides Run free audit

+1 free check

Welcome to Shipwork

Sign in free and your next check this month is on us.

Continue with GoogleContinue with email

We email you a 6-digit code. No password.

No card needed. Reports and watches follow you to any device.

Which pages is your sitemap missing?

Paste your address. This reads your sitemap and crawls a bounded sample of the site, then compares both directions: pages that exist but are absent from the sitemap, and sitemap URLs that no longer answer 200.

Reads /sitemap.xml and a bounded sample of the site. Nothing is stored.

How this works

Your sitemap is read the way a crawler reads it, then a bounded sample of the site is crawled through its own links. A page the crawl reached but the sitemap does not list is reported as missing from the sitemap; a sitemap URL that does not return 200 is reported the other way. This is the reverse of the orphan-pages check, which starts from the sitemap and looks for pages nothing links to.

What the sitemap diff compares

The tool reads your sitemap and crawls a bounded sample of the site through its own links: the homepage, the links on it, and the links on a set of sampled pages. Every URL it reaches is compared with the sitemap list in both directions. A reached page that the sitemap does not list is reported as missing from the sitemap.

In the other direction it takes a spread sample of the sitemap's own URLs and requests them, without following redirects, to see whether they still return a 200. A URL that now answers with a redirect, a not-found response or no response at all is reported so you can update or remove it. The sample is spread rather than taken from the top, so a whole section is not missed.

How to read the two lists

A page missing from the sitemap is not necessarily broken; it may load perfectly for visitors. The point is that the file search engines read does not mention it, so discovery depends on links alone. A template change can drop a whole section from the sitemap while the pages themselves keep answering. It may also be a page you do not want indexed, in which case a noindex is the right fix.

A sitemap URL that is not a 200 sends crawlers to an address that redirects or fails. The reverse check is a sample, so a clean result covers the URLs checked, not every URL in the file. The missing list is also bounded, and a page that nothing links to and the sitemap omits cannot be seen by this method.

Why a sitemap drifts from the site

A redesign or a platform move often rebuilds templates and the sitemap together, and one side can fall behind. A new page type may be excluded by a setting, or an old page type may still be listed after its pages were retired. A sitemap index that lists only one child file can hide every other section without any error.

When the file and the site disagree, crawlers spend budget on addresses that no longer work and find newer pages more slowly. That is why the check reports both directions: adding the missing pages and removing the dead ones are two halves of the same repair, and fixing one side alone leaves the other mismatched. The mismatch is usually cheap to fix once you know the cause.

Fix the sitemap and the site together

Add the missing pages to whichever system generates the sitemap, so a future rebuild keeps them, rather than editing the file by hand. For a sitemap URL that is not a 200, either update it to the live address, remove it, or restore the page if it should still exist. If a page moved, redirect the old address and list the new one.

Confirm the sitemap is declared in robots.txt and reachable, then run the diff again. Correct the source, not just the output file, so the same mismatch does not return on the next deploy. If the sitemap is generated by a platform setting, change the setting rather than the file it produces. A count limit that is exceeded can silently drop the tail of the file.

When to use related checks

Sitemap check validates the file itself, and orphan pages starts from the sitemap to find pages nothing links to, the other side of this diff. Index signals compares sitemap entries with noindex and canonical signals. This page reports the difference between the file and the site, while the index signals check confirms whether the listed URLs agree with the page-level rules.

For the generator side, use sitemap generator. See the sitemap and Search Console guide for fetch errors, and the fixes for a missing sitemap and a sitemap that will not fetch. Keep the file inside the count limits the platform supports, and split a large sitemap into an index of child files so no section is dropped by a size limit.

Questions

Which direction matters after a redesign?
Pages that exist and are crawlable but are missing from the sitemap. A redesign can drop a template out of sitemap.xml while the pages still answer 200, so search engines only find them through links, later and at lower priority.
What does "missing from the sitemap" mean?
The URL is not listed in sitemap.xml at all. The page may be perfectly reachable; the point is that the file search engines read does not mention it, so discovery depends on links alone.
Why would a sitemap URL not return 200?
The page was deleted, moved, or renamed and the sitemap is stale. A sitemap that lists pages that 404 (or redirect) sends crawlers to dead ends and wastes the budget you want spent on real pages.
How many pages does it check?
A bounded sample: the homepage, the links reachable from it and from a set of sampled pages, and a spread selection of sitemap URLs. It is enough to catch a template or a whole section that fell out of the sitemap, not an exhaustive export of every URL.
How do I find pages missing from my sitemap?
Enter your site address. The tool reads the sitemap, crawls a bounded sample of the site through its links, and lists reachable pages that the sitemap does not mention. It also checks a sample of sitemap URLs to see which ones no longer return a 200.
What does it mean if a URL in my sitemap is not a 200?
The page was deleted, renamed or moved while the sitemap stayed stale. A redirect or an error in the sitemap wastes crawl budget on addresses that do not resolve. Update the entry to the live URL, remove it, or restore the page. Check the URL directly to see where it leads.
How many pages does the sitemap diff check?
It is bounded. It crawls a sample of pages and checks a spread selection of sitemap URLs for a 200. That is enough to catch a template or a whole section that fell out of the file, not an exhaustive export of every URL on the site or a guarantee about the ones it did not reach.
Is a page missing from the sitemap a problem if it loads?
The page can still be found through links, so it is not broken. The risk is slower, lower-priority discovery. Listing your important indexable pages in the sitemap is how you tell crawlers they exist and should be checked, and it also carries the last modification date when one is provided.
Does the sitemap diff use Search Console data?
No. It reads your public sitemap and crawls the site from outside your network. It does not report whether a URL is indexed, and it does not connect to any search account. It compares the sitemap with the site itself, not with the index.