llms.txt Adoption: How Many of the Top 10,000 Sites Have One?

Key findings

• 16.1% of the top websites serve an llms.txt (881 of 5,480), in our October 28, 2026 crawl of the Tranco top 10,000.
• The top 1,000 lead: 22.5% have one, against 15.4% of sites ranked 1,001 to 10,000.
• About half are malformed. 414 of the 881 files (47%) pass the llmstxt.org structure. The usual fault is prose inside a link section (409 files).
• 11.3% of sites answer /llms.txt with a web page, usually their homepage, which looks like a file in a browser and is nothing to a parser.
• The typical file is small: a median of 7,284 characters and 33 links. 59 files run past 100,000 characters.
• 36 sites publish an llms.txt and block GPTBot, and 4 block OAI-SearchBot, so OpenAI’s own search crawler may not fetch the guide written for it.

Adoption falls with rank, from 22.5% in the top 1,000 to 10.6% in the 9,001–10,000 band, though not smoothly: the 2,001–3,000 band matches the top. Software and infrastructure companies make up much of the early list. In the top 200 we could read, adopters include cloudflare.com, fastly.com, wordpress.org, adobe.com, digicert.com, opera.com, sentry.io, intuit.com, samsung.com, dropbox.com, gravatar.com, kaspersky.com, ubuntu.com, shopify.com.

What the files look like

The llmstxt.org format is short: one H1 with the site’s name, an optional blockquote summary, optional paragraphs, then H2 sections that contain only lists of links. The three faults we found most:

• Prose or plain bullets inside an H2 section (409 files). The spec reserves those sections for - [name](url): notes items, so a reader that follows it skips the rest.
• More than one H1 (90 files), usually a long document pasted in whole.
• No H1 at the top (42 files), often because the file opens with its own URL or a comment.

Does anyone read it?

Not in a way the major assistants have confirmed. Google says plainly that you “don’t need to create new machine readable files, AI text files, or markup” to appear in AI Overviews or AI Mode (Google Search Central), and the crawler documentation from OpenAI, Anthropic and Perplexity doesn’t mention llms.txt. Its clearest readers today are coding agents and documentation tools. So the 16.1% figure measures how many teams have decided it’s worth twenty minutes, not a ranking factor. Our guide What is llms.txt covers what to put in one.

If you publish one

1. Check it returns the file. Open /llms.txt and view the source. If you see your homepage’s HTML, you don’t have one.
2. Keep sections to links. Put any explanation in the summary or in paragraphs before the first H2.
3. Keep it an index. A list of the pages that answer buyer questions, with a note on each, beats a copy of your site.
4. Let the crawlers in. A file written for language models helps little if robots.txt blocks the crawlers that would fetch it. See which sites block AI crawlers.
5. Validate it. The llms.txt validator is the same code we used here.

Method and limits

We took the top 10,000 domains of the Tranco list GQ3LK (a research ranking that combines Chrome UX Report, Cloudflare Radar, Cisco Umbrella, Majestic and Farsight data over September 28 to October 27, 2026). On October 28, 2026 our crawler requested each domain’s homepage over HTTPS, following redirects. Many Tranco entries are CDN, API or tracking domains with no website: 2,155 gave no HTTP answer at all and 1,415 answered 403. We kept the 5,480 domains whose homepage returned an HTML page with status 200, counted once per final hostname after redirects.

For each site we requested /llms.txt on the final hostname. We counted a site as having one when the answer was 200, not HTML, and read as markdown (a heading or a markdown link near the top). 617 sites returned HTML, 44 returned something else (JSON error bodies, mostly), 3,789 returned 404 or another 4xx, and 149 failed. Structure was checked with the validator behind our llms.txt validator; “passes” means no errors against the llmstxt.org format, warnings allowed. We also requested /llms-full.txt.

Our crawler identified itself as Mentioned-Research/1.0. Sites that refuse unknown bots at the firewall are missing from the sample, and those sites are probably stricter with AI crawlers too, so treat every blocking rate here as a floor rather than an estimate of the whole web. The figures describe popular websites, not the web at large, and one day in October 2026.

Cite this

You are welcome to quote and chart these figures with a link back. Suggested citation: Mentioned, “llms.txt Adoption: How Many of the Top 10,000 Sites Have One?,” October 28, 2026, https://mentioned.at/learn/llms-txt-adoption. Data and text are published under CC BY 4.0.

Questions

Run a free check