llms.txt
llms.txt is a proposed standard file at your domain root that gives AI systems a curated, markdown map of your most important content, helping them find and understand your best pages.
Also known as: LLMs.txt
llms.txt is a guided tour of your site for AI. Proposed at llmstxt.org, it's a
markdown file at example.com/llms.txt that lists and describes your key pages in a
clean, machine-friendly format — a curated index pointing answer engines straight at
your most citable content, rather than leaving them to crawl everything blind.
Where robots.txt controls what crawlers may access, llms.txt is about helping them understand what matters — a complement to the access pillar. Adoption is still emerging and not all engines consume it yet, so treat it as a low-cost, forward-looking signal rather than a guaranteed ranking factor. It's especially natural for content-heavy sites that want to surface their best answer-first pages.
Example. This site publishes its own /llms.txt, listing the Canon pillars, articles, and this glossary with one-line descriptions — a single file that hands an AI system a structured map of where the good answers live.
Relevant pillar
Related terms
- robots.txtrobots.txt is a plain text file at the root of your domain that tells crawlers which user-agents may access which parts of your site, and is how you allow or block AI crawlers.
- Answer EngineAn answer engine is a search system that responds to a question with a direct, synthesized answer instead of a list of links, usually citing the sources it drew from.
- GPTBotGPTBot is OpenAI's web crawler that gathers content to train its models, identified by the GPTBot user-agent and controllable through your robots.txt file.