ShipworkSite inspection
llms.txt builder

Crawlability

PaginationAre paginated pages set up right?Internal searchIs site search eating crawl budget?URL parametersAre parameters creating duplicates?Log analyserWhat does Googlebot actually crawl?Robots builderNeed a robots.txt file?Soft 404sAre pages “not found” but returning 200?JavaScriptCan crawlers see it without running JS?FreshnessIs my site quietly going stale?Robots.txtDoes robots.txt say what I think?

Indexing

Sitemap lastmodAre my sitemap dates valid?SitemapIs my sitemap actually fetchable?IndexingIs Google allowed to index my pages?CanonicalsIs Google indexing the wrong URL?Index signalsDo sitemap and index signals agree?Sitemap generatorNeed a sitemap.xml for my site?Bulk URL statusWhere does each of these URLs really land?Redirect builderNeed the redirect rules?RedirectsAre my URLs answering directly, over HTTPS?

On-page

Keyword ideasWhat are people searching for?SERP previewHow does my page look in Google?Content qualityAre your pages too thin or too alike?Image weightAre images slowing the page?CannibalizationAre pages competing with each other?On-page checkDoes the page use its target phrase?ImagesAre my images accessible and loading?Meta tag builderWhat should my title and share tags say?DuplicatesDo my pages compete for one query?

Links

Orphan pagesWhich pages can no link reach?Link graphHow deep do your pages sit?Anchor textDo links say what they point at?Outbound linksAre external links still alive?Broken linksAre internal links sending visitors nowhere?

Structured data

Rich resultsIs my markup eligible for a rich result?Structured dataIs my product schema valid?Schema coverageDo my key pages carry structured data?Social previewHow does my page look when shared?Schema vs pageDoes markup match the page price?Schema builderNeed valid JSON-LD?

International

Hreflang sitemapDo page and sitemap hreflang agree?HreflangDo my language versions link back?Hreflang builderNeed the hreflang tags?Page DoctorWhy is this page not doing well?Redirect planMy site has dead links — what redirects do I write?Sitemap diffIs anything missing from my sitemap?Robots simulatorWhat does my robots.txt actually block?Can Googlebot?Can Googlebot fetch this URL?
Every error, explainedGuides
Pricing Learn Error guides Run free audit

+1 free check

Welcome to Shipwork

Sign in free and your next check this month is on us.

Continue with GoogleContinue with email

We email you a 6-digit code. No password.

No card needed. Reports and watches follow you to any device.

Generate an llms.txt for your site

Name the site, write one line about it, list the pages that matter. This produces a plain-text file to serve at /llms.txt so an assistant can describe you accurately.

Manual generation runs in your browser. Sitemap lookup reads public pages.

After generating

Serve the file at /llms.txt on the exact host, then run the llms.txt check. It fetches the file from outside your network and reports whether it exists, whether it has real content, and how many pages it links to. Keep it honest: it describes pages that are already public and readable.

What the llms.txt generator writes

This browser-based form takes a site name, an optional one-line description, key URLs, and optional notes for AI. The site name becomes an H1, the description becomes a blockquote, and the URLs become a Pages section with Markdown links. Each URL line can include a title and description separated by vertical bars. The tool includes example presets for a store and a content site, and nothing is sent to a server while you generate the file.

The result is a downloadable or copyable Markdown text file. It is not a crawler, content audit, robots.txt editor or publishing integration. The generator does not discover your pages, verify the URLs, check whether their content is public, or decide which pages matter. You choose the source information and should inspect every line before you use it.

How to structure useful entries

Start with a site name and a sentence that says what the site is, not what you hope an assistant will say. Add a short list of canonical pages with descriptive titles. For example: - [Returns](https://example.com/pages/returns): Return window, eligible items, and how to start a return.. Group the links by topic if the list is long enough to need sections, and keep each description specific to its destination.

Optional notes can clarify facts an assistant might otherwise misread, such as currency, shipping exclusions, or how product availability is shown. Keep notes short and verifiable. Do not put private customer information, credentials, promotional claims without evidence, or instructions that conflict with the linked page. The file should help navigate existing public content, not replace it.

Review and correct generated output

The form accepts one URL per line, with optional title and description fields separated by vertical bars. A missing title falls back to the URL itself. If a URL has more than one description separator, the remaining pieces are joined into the description. The generator builds a # heading, optional > summary, a ## Pages heading and the link entries; optional notes appear under ## Notes for AI.

Before publishing, look for accidental placeholder domains from the examples, duplicate links, redirects and stale paths. Open each URL in a private browser window. Confirm that its title and summary match the destination, the canonical page is the one you intend, and the page is readable without a login. Remove any generated example values you did not replace.

Publish at the site root

Save the output as llms.txt and serve it at https://example.com/llms.txt on the intended hostname. Uploading it to a page editor or a subdirectory does not put it at the conventional root path. Your host or CMS may have a file upload, static asset or deployment mechanism; choose the method that serves a plain-text response and does not require authentication.

After publishing, fetch the URL directly rather than trusting a successful download from the generator. Check the exact domain and any www alias, then run the llms.txt check. That checker confirms external retrieval, meaningful length and a URL count; it does not validate every destination. Use your own link review for that final step.

Set realistic expectations

The format is optional and is not uniformly adopted by AI products. A valid generated file cannot force a crawler to read it, fix a blocked product page, or guarantee citation. Make the underlying pages accurate and accessible first. Robots.txt and network controls govern different aspects of crawling; llms.txt is a map or summary, not permission and not security.

For Shopify examples, see the llms.txt on Shopify guide. If the concern is a bot being blocked, check AI crawler access separately. The generator and checker answer different questions: one creates a draft file from your inputs; the other fetches the version currently published on your site.

Questions

What is llms.txt?
A Markdown file at /llms.txt that names the site, gives a one-line summary, and links the pages that matter so AI tools can describe and cite it accurately. The format is described at llmstxt.org: an H1 title, an optional blockquote summary, then sections of links.
Do assistants actually read it yet?
Some do, not all, and it is not a ranking or indexing signal. It is a young convention: cheap to add, no downside, but do not expect it to fix content a crawler cannot otherwise read.
Where do I put the file?
At your site root, served over HTTPS at /llms.txt, the same place robots.txt lives. It is plain text and should be fetched without a login.
How is this different from robots.txt?
They do different jobs. robots.txt controls which paths a crawler may fetch. llms.txt describes what the site is and points at the pages worth reading. A site can allow every crawler and still have no llms.txt, or the reverse.
How do I create an llms.txt file for my website?
Enter your site name, an accurate one-line summary and important public URLs. Optionally add notes that clarify facts already supported by your pages. Generate the Markdown, replace any example values, inspect all links, and publish it as llms.txt at the root of the hostname you want checked.
What format does an llms.txt file use?
This generator uses a Markdown outline: an H1 site name, an optional blockquote summary, a Pages section containing links and optional descriptions, and an optional Notes for AI section. The convention is still evolving, so keep the file simple and confirm it remains useful to your intended readers and tools.
Does the generator find my important pages automatically?
No. You supply the URLs and optional titles and descriptions. The tool does not crawl the site or verify the destinations. Use your sitemap or navigation to choose relevant public pages, then open each generated link and confirm it is current and canonical before publishing.
Where do I upload the generated llms.txt?
Serve it at the site root, such as https://example.com/llms.txt, over HTTPS and without a login. The exact upload method depends on your host or CMS. After deployment, request that URL directly; a file downloaded to your computer or uploaded to a subfolder is not the public root file.
Should I add every page to llms.txt?
Usually not. The file is most useful as a concise guide to the pages that explain your organization, products, policies or expertise. A long uncurated list is harder to maintain and can include stale or low-value URLs. Pick a manageable set and update it when those pages change.