Find pages with duplicate content or very little text

Duplicate content finder: a worked example

Suppose /products/linen-shirt-blue and /products/linen-shirt-green repeat the same description, with only the colour changed. The duplicate content finder compares body wording across sampled pages after removing common boilerplate. If the remaining wording meets the near-duplicate threshold, the pair is a prompt to review the pages, not an instruction to delete either product.

If each colour page helps shoppers choose a different item, keep both and add useful details specific to that variant: colour, photographs, availability and care information where they differ. If the URLs are two copies of the same item, decide which URL should remain and consolidate deliberately. Do not redirect a distinct product merely because its description is similar.

After editing, run the same domain again and check whether both URLs were included in the sample. Review the reported pair alongside the rendered pages. A pair disappearing from a bounded sample does not prove the entire site is free of duplicates. Use canonical check to inspect declared preferred URLs and duplicate title and description check for repeated metadata.

What the thin content checker measures

The thin content checker counts the visible words on each page it reads and flags pages under 250 words as thin, excluding the home page and pages that did not return a normal response. The free run reads a sample of up to 20 pages. It then compares the words of those pages to find near-duplicates, stripping out common boilerplate first.

Boilerplate is identified as any word that appears on 70 per cent or more of the sampled pages, which is usually navigation, header and footer text. The checker ignores pages with fewer than 20 remaining words and reports pairs whose remaining wording overlaps by at least 80 per cent. It lists up to three such groups of pages.

How to read a content quality result

A thin warning means those pages carried under 250 words of visible text. That is a shortlist for review, not proof that a page cannot rank: some legitimate pages are short, and some short pages answer a question completely. A near-duplicate error means two or more sampled pages used almost the same wording once boilerplate was removed.

The check reads the HTML response, so text that only appears after scripts run is not counted. It cannot judge relevance, usefulness or search intent, and it does not measure readability. A clean result means the samples were distinct and substantial; it does not prove every page on the site is, because the crawl is bounded.

Common causes and why they matter

Thin pages tend to be templates: tag archives, filtered collections, author pages and product variants that differ only by a single attribute. Near-duplicates appear when the same description is reused across colour or size variants, when a feed copies supplier text to many items, or when location or campaign pages share one body with the place name swapped.

A thin page rarely answers a search well, and a page that duplicates another competes with its own site for the same query. When two near-identical pages exist, a search engine chooses one, and it may not be the one you would pick. The signals that could have made a single strong page are split between them.

How to fix thin and duplicate content

For a thin page, decide whether it has a purpose. If it does, expand it with information only that page can offer. If it does not, merge it into a stronger page with a redirect or remove it. For near-duplicates, consolidate the best content into one URL and redirect the others, or add a canonical that points at the version you want indexed.

Where variants must stay separate, write distinct copy for each rather than repeating one block. On a store, edit the product description in the catalogue so future imports do not overwrite your change, and check whether the variant template is echoing the parent description. Re-run the checker after editing to confirm the counts improve.

When to use related checks

Use duplicate content when the concern is identical titles or meta descriptions rather than body copy. Use soft 404 check when a page returns a normal response but reads as empty or missing.

The duplicate content guide covers consolidation in more detail. Thin content and duplicate content are related but separate signals, so read the report that matches the problem instead of treating every warning as the same issue.