Before anything else, search for the exact URL rather than a phrase from the page. If the URL matches a different page you own, you have a duplicate or canonical problem, not an indexing problem. If nothing matches, keep going.
1. Is it blocked from being crawled?
robots.txt controls crawling, not indexing, but it comes first because a blocked page cannot deliver any of the other signals. A Disallow rule that matches the path, whether you intended it or not, stops the crawler before it sees the page at all.
2. Is it marked noindex?
There are two places noindex hides: a meta robots tag in the HTML and an X-Robots-Tag response header. The header is the one people miss, because it is not in the page source and is often set at a CDN or proxy that was configured for staging. A noindex only works if the page can still be crawled, which is why steps one and two are separate.
3. Does the canonical point somewhere else?
If the canonical on the page names a different URL, you have told search engines which version to index, and it is not this one. That is correct for a genuine duplicate and wrong when a plugin or tag is adding a canonical you did not intend.
4. What status does the URL actually return?
A 404 or 410 cannot be indexed. A redirect means you should be looking at the destination. A 200 that renders an empty or error message is a soft 404 and is treated as thin content even though the status says success.
5. Can anything reach it?
A page with no internal links is an orphan. Crawlers find it far less often, and it collects no internal authority, so even a good page underperforms. A page in the sitemap but linked from nowhere is a common form of this.
6. Is it in the sitemap, and is the sitemap healthy?
The sitemap is a hint, not a command, but it is the clearest signal you can send that a URL is worth fetching. A sitemap that 404s or lists dead URLs sends the opposite message.
The tool that checks all six at once
The indexing check reads robots.txt, the meta robots tag, the X-Robots-Tag header, the canonical, the response status and the sitemap for one URL, from outside your network, and reports which of the six is the reason.
- Indexing checkResolves the blocks and noindex signals for a URL.
- robots.txt, explainedHow the rules are resolved and what they really block.
- Canonical tagsIf the canonical is why the wrong URL is indexed.
- Orphan pagesSitemap URLs no internal link points at.
- XML sitemapsValidating the file that tells crawlers where to look.