Allow the search crawler
OpenAI distinguishes OAI-SearchBot, used to surface websites in ChatGPT Search, from GPTBot, used to crawl content that may be used to train models. Its documentation says sites that opt out of OAI-SearchBot will not be shown in ChatGPT Search answers, though they may still appear as navigational links. Check your live robots.txt for a specific group that disallows it:
User-agent: OAI-SearchBot Disallow: /
If Search visibility is a goal, remove or narrow that block. You may still choose a separate policy for GPTBot. OpenAI also publishes IP ranges for the search crawler and recommends allowing its requests at the network layer; a permissive robots.txt is ineffective if your CDN returns a challenge or 403. Read the current OpenAI crawler overview before changing access rules.
ChatGPT-User is different again: it may fetch a page after a person asks ChatGPT to open it, but OpenAI says it is not used to determine whether content appears in Search. That user-triggered access is not a dependable substitute for OAI-SearchBot.
Make source-worthy pages
When an assistant answers a question with web sources, the useful target is a page that directly supports a specific answer. Build pages around questions your customers actually ask, and make the evidence easy to inspect:
- Answer the core question near the beginning, then explain the conditions, exceptions and steps that change the answer.
- Publish first-hand details you can substantiate: tested processes, original measurements, product specifications, policies and examples.
- Name the organization responsible for the page, provide a way to contact it, and show authorship or review information where appropriate.
- Keep claims current. Put dates on time-sensitive policies and update the source page when the underlying details change.
- Use descriptive headings, meaningful link text and HTML text that is available without a login. Give images useful alt text where the image conveys information.
- Link to primary evidence, such as your policy, product specification, official documentation or research method. Cite sources for claims that readers should be able to verify.
These are sound publishing and accessibility practices, not a secret ChatGPT ranking formula. OpenAI does not document a guaranteed citation score, special schema, preferred word count or paid inclusion path for ordinary web pages.
Keep the page reachable
- Serve the answer at a stable, canonical URL and include it in a working XML sitemap.
- Allow OAI-SearchBot in robots.txt if you want Search discovery, and ensure the CDN, firewall and origin can serve it.
- Return the page content to an unauthenticated visitor; avoid requiring client-side interactions that hide the important material.
- Use ordinary internal links so readers and crawlers can find the page from relevant parts of your site.
- Check that the page is not marked noindex and that the canonical points to the URL you want indexed.
Structured data can describe entities and products when it matches visible page content, but it does not buy an AI citation. Google’s structured data policies likewise emphasize accurate, visible, relevant markup; no markup guarantees special display. Avoid adding FAQ or other schema solely on the assumption that ChatGPT will prefer it.
Measure real referrals
Use your analytics and server logs to look for visits referred from ChatGPT, then check the landing page and the visitor’s path. Referral reporting can vary by product, link handling and analytics setup, so absence of an obvious referrer is not proof that a page was never seen or cited. Keep the crawler log distinct from human referral traffic: a crawler hit is not a citation or a customer visit.
Compare useful outcomes over time: qualified visits, product views, enquiries and conversions from the landing pages that received referrals. Do not treat a one-off citation as a stable ranking or promise it will persist; answers and source selection can change.
Avoid citation promises
Be cautious of vendors promising guaranteed inclusion, “AI authority” scores, or a required llms.txt submission. A file that explains your site may help a human or tool navigate it, but OpenAI’s crawler guidance does not make llms.txt a requirement for ChatGPT Search. Do not invent author credentials, fake customer reviews, or publish pages made only to repeat keywords. These practices erode trust and cannot guarantee that a page will be used.
For bot roles and robots.txt controls across providers, see AI crawler user agents. For the decision to block model-training crawling while keeping search access, read how to block GPTBot.
Shipwork can verify the public robots.txt side of crawler access; citation selection itself is controlled by the search product and is not something a site checker can guarantee. Free, no account, no signup. Paste your store address.
Check crawler accessQuestions
How do I get my business cited by ChatGPT?
Do I need to allow GPTBot to appear in ChatGPT Search?
Does adding llms.txt get a site cited?
Does structured data help ChatGPT cite my page?
Keep reading