Is Google actually allowed to index your pages?

A single noindex tag, a stray robots header or a page-level block can remove a page from search without changing how it looks to you at all. This reads your key pages the way a crawler does and reports what it is told.

A bounded sample of your key pages plus robots.txt, fetched from outside your network. Nothing is stored.

How this check works

Shipwork crawls a sample of your key pages and reads the raw HTML and response headers — the same signals a crawler sees before it runs any JavaScript. It reports pages marked noindex, pages returning errors to an outside request, and robots.txt rules that block crawlers from the whole site.

Questions

Why are my pages missing from Google?
The common silent causes are a noindex tag, an X-Robots-Tag header, a robots.txt rule blocking crawlers, or a page that returns an error to an outside request. All are invisible in your own browser.
Does this work on any platform?
Yes. It reads HTML and response headers, so it works on Shopify, WooCommerce, Wix, Squarespace and hand-built sites alike.
What is the difference between noindex and a robots.txt block?
noindex tells engines not to list a page they may still crawl; robots.txt tells them not to crawl it at all. Either can remove a page from search.

Keep reading