robots.txt Shopify: What to Allow

robots.txt for Shopify: the lines that matter

Shopify writes most of your robots.txt for you, and lets apps and snippets rewrite it without asking. One wrong Disallow can hide your products from Google; one missing Allow is harmless but sloppy. Here is the template that works, line by line, and the mistakes that actually hurt stores.

By Hazzy · Software engineer, former SEO specialist

The template that works

User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkouts/
Disallow: /checkout
Disallow: /carts
Disallow: /account
Allow: /products/
Allow: /collections/
Allow: /pages/
Allow: /blogs/

Sitemap: https://yourstore.com/sitemap.xml

The shape is deliberate: transactional and account paths closed, content paths open, sitemap declared with the exact live URL. Replace the domain, keep the shape.

Closed: logins, carts and internal search. Open: everything a shopper (or Google) should find. One wrong line in the left column hides products.

What stays disallowed

Path	Why closed
/admin, /account	No crawler business behind logins; login pages dilute demand
/cart, /carts, /checkout*	Transactional dead-ends that multiply URL space
/orders	Private order-status pages with zero index value

What must stay crawlable

Products, collections, pages and blogs are the indexable surface of the store: each needs an explicit Allow so no broader rule can accidentally swallow it. The Sitemap line must name the exact root URL Google should fetch; a stale domain here is one of the top “could not fetch” causes. Anything AI-specific lives in named-agent groups alongside this base, covered in the AI crawler guide.

The mistakes that hurt

Disallow: / under User-agent: *: total blackout, usually left over from a staging or maintenance window. Check first, always.
Blocking /products/ or /collections/: sometimes added by SEO apps during setup. Products vanish from the index while the store looks untouched.
Over-broad prefixes: Disallow: /s blocks everything containing /s. Prefer full path segments with trailing slashes.
Edits nobody remembers: apps and copied snippets rewrite the template silently. Re-read the live file after every theme or app change.

Test the file, not your memory

Fetch /robots.txt fresh and test your top product and collection URLs against it. Memory of what the file “should” contain is how stores stay blocked for months.

Deeper: robots.txt for AI crawlers · sitemap pillar · Related guide: could not fetch

Find out if this is happening to you

Shipwork fetches your robots.txt and sitemap together from outside your network, verifying the sitemap reference resolves and nothing crawlable is blocked. Free, no account, no signup. Paste your store address.

Check my sitemap

Questions

Where is the Shopify robots.txt file?

Served at yourstore.com/robots.txt, generated from a template you can edit. Read the live file, not the template. Apps can change the output.

What should robots.txt be on Shopify?

Shopify generates a sensible robots.txt for you. The lines that matter are the ones you add: keep products, collections and pages crawlable, disallow /cart, /orders and /search, and keep a Sitemap line pointing at your live domain. Then read the served file, because apps can rewrite it.

Should I block the search (/search) pages?

Yes: internal search results add URL space without index value. Disallow /search while keeping products and collections open.

Why does my sitemap line matter so much?

It tells crawlers exactly which file to fetch. A stale domain or path here is a top-three “could not fetch” cause.

How do AI agent rules fit in?

Named-agent groups sit alongside the general rules; the most specific group wins per agent. The AI crawler guide covers the nine agents that matter.

Keep reading

Shopify Sitemap Fetch Errors in Search Console Hreflang Sitemap Example: Two Localised Pages Crawl Budget for Shopify: Get Products Indexed Google Merchant Center Requirements for Shopify