What the error means
Search Console reports pages excluded because robots.txt disallows their path. The pages are fine — the file is lying to the crawler about them, usually by accident: an over-broad Disallow prefix, a staging leftover, or an app that edited the template.
Why Google rejects it
robots.txt is prefix-matching and the longest matching rule wins. Disallow: /s does not mean "block /search" — it blocks every path starting with s, including products. A line written for staging or for one app’s endpoints can quietly swallow product paths, and because the pages return 200 in a browser, nothing looks wrong.
How to fix it
- Read the live file at yoursite.com/robots.txt — not the template you remember. Apps rewrite it silently.
- Test the affected URL against the rules: find every Disallow whose prefix matches, and remember the most specific rule wins.
- Replace over-broad prefixes with full path segments (Disallow: /search/ not /s), and never leave a bare Disallow: / from a maintenance window.
- On Shopify, keep /admin, /cart, /checkout blocked and /products/, /collections/, /pages/ explicitly allowed.
- Recheck after any theme or app change — the file is edited by many hands.
Shipwork fetches your robots.txt and your sitemap together, resolves the rules for each sampled URL the way a crawler does, and reports which pages are blocked and by which line. Free, no account, no signup — paste your store address.
Check my storeGo deeper: Shopify robots.txt, line by line
Other errors like this