What Amazonbot does
Crawls the web for Amazon services, including answers from Alexa and other AI features.
Amazon documents that Amazonbot respects robots.txt.
How to block Amazonbot in robots.txt
Add these two lines to the robots.txt file at the root of your domain: User-agent: Amazonbot and, on the next line, Disallow: /
To block only part of the site, replace / with a path such as /private/. Rules apply per host, so a subdomain needs its own robots.txt.
How to allow Amazonbot
If you have no rule for Amazonbot and no blanket User-agent: * Disallow: /, it is already allowed. To allow it explicitly while blocking other bots, add User-agent: Amazonbot followed by Allow: /
Check it worked
- Publish the updated robots.txt and open it in a browser at /robots.txt to confirm the live file has the rule.
- Run the free AI crawler check: it reads your live robots.txt and reports whether Amazonbot is allowed or blocked.
- Look for Amazonbot in your server or CDN logs over the following weeks to see whether requests stop.
robots.txt is a public request, not access control. Amazon publishes its own documentation for Amazonbot; confirm the current token and behaviour there, since operators change them.
- Check which AI bots can read your siteTests your live robots.txt against every major AI agent.
- Block AI crawlersGenerate robots.txt rules for training bots, search bots or both.
- Every AI crawler user agentThe full list with operators and purposes.
- GPTBot vs ChatGPT-UserWhy OpenAI uses more than one agent.
Related explainers
- Why is my page not indexed
- robots.txt vs noindex
- Missing canonical
- Free technical SEO audit
- AI crawler user agents
- GPTBot user agent
- OAI-SearchBot user agent
- ChatGPT-User user agent
- ClaudeBot user agent
- Claude-SearchBot user agent
- PerplexityBot user agent
- Google-Extended user agent
- Applebot-Extended user agent
- CCBot user agent
- Bytespider user agent
- Meta-ExternalAgent user agent
- SEO MCP server
- 404 vs 410
- hreflang vs canonical
- What is a canonical tag