What llms.txt is, and why publish one
Search is shifting from a list of links to a single synthesized answer, and the engines that
write those answers — ChatGPT, Perplexity, Claude, Google's AI overviews — decide what to read
and cite from your site. llms.txt is a small, emerging convention
that helps: a Markdown file at your domain root that curates your most important, canonical pages
in a clean, link-first format a model can parse without wading through navigation and ads.
It is not an official standard and no engine is known to require it, so treat it as a cheap hedge,
not a silver bullet — publish it alongside the signals that matter more (crawlability for AI bots,
structured data, and answer-first content). The format is deliberately simple: an H1 title, an
optional summary blockquote, and one or more ## sections of
Markdown links with short notes. This tool produces exactly that.
After you publish it
- Serve the file at
https://yoursite.com/llms.txtas plain text. - Keep the link list short and canonical — your best pages, not every page (that is what your sitemap is for).
- Audit it for link rot the same way you audit your sitemap; a stale map is worse than none.
- Want it scored? Our citation-readiness checker and the AEO audit endpoints validate llms.txt structure and section coverage.
Sources & further reading
The conventions and standards this tool builds on:
- RFC 9309 — Robots Exclusion Protocol (IETF, 2022) — the companion
robots.txtstandard that governs AI-crawler access. - JSON-LD 1.1 (W3C Recommendation) — the structured-data layer that pairs with llms.txt for machine legibility.
- Large language models (Wikipedia) — the readers llms.txt is written for.