llms.txt explained (2026): the robots.txt for AI — and how to write one

You know robots.txt and sitemap.xml. Meet the new one: llms.txt — an emerging, robots.txt-style file that helps AI models find and understand your content. It’s one of the cheapest, most future-friendly moves in the GEO / AI-visibility toolkit, and it takes minutes. Here’s what it is, how to write one, and — honestly — what it does and doesn’t do. (This site runs one.)

What it is

llms.txt lives at your domain root (yoursite.com/llms.txt) and gives large language models — ChatGPT, Perplexity, Gemini and the crawlers behind them — a clean, curated, markdown map of your most important content. Where robots.txt sets access rules, llms.txt offers a human-readable guide: here’s what this site is, and here are the key pages worth reading.

File	Purpose	For
robots.txt	Access rules (allow/disallow)	Crawlers
sitemap.xml	List of URLs to crawl	Search engines
llms.txt	Curated markdown map + descriptions	AI / LLMs

They’re complementary — keep all three.

How to write one

It’s plain markdown. A minimal, effective structure:

H1 — your site name.
A short blockquote summarising what the site is.
Sections with bulleted links to your most important pages, each with a one-line description of what it covers.

Keep it curated — point models at your best, most useful content, not every URL (that’s what sitemap.xml is for). Optionally add an llms-full.txt with more detail. Then deploy it so it’s reachable at yoursite.com/llms.txt. No special tooling required.

The honest part: what it does and doesn’t do

llms.txt is young and not yet universally honoured. Adoption is growing and several tools reference it, but treat it as a low-effort, future-friendly bet — not a switch that turns on AI traffic. The real drivers of AI visibility are still:

being genuinely the right answer to a specific question,
clear, extractable content a model can quote,
and presence where models source (covered in how to get AI traffic to a new website).

llms.txt sits on top of that — a sensible, cheap signal, not a substitute for the rest of GEO / AI SEO.

Where it fits

It’s one technical piece of the bigger picture: position to be the answer, write to be cited, be present where models read, run llms.txt, and measure AI referrals in GA4 (Search Console can’t see them). The full method: Generative Engine Optimization (GEO).

The takeaway

llms.txt = a curated markdown map of your site for AI, at yoursite.com/llms.txt.
Complements (doesn’t replace) robots.txt + sitemap.xml.
Curate your best pages; keep it short; optionally add llms-full.txt.
Young standard — a cheap, future-friendly bet, not a magic switch.

It costs ten minutes and signals you’re building for the AI-search era. Add it, then put your real effort into being the answer — the full playbook is in GEO.

Frequently asked questions

What is llms.txt?

llms.txt is an emerging, robots.txt-style file you place at the root of your domain (yoursite.com/llms.txt) to help large language models find and understand your most important content. Instead of access rules like robots.txt, it provides a clean, curated map — in plain markdown — of your key pages and what they cover, so AI tools like ChatGPT, Perplexity and Gemini can more easily locate and cite the right parts of your site. It is part of the wider GEO / AI-visibility toolkit, alongside being the clear, citable answer to the questions people ask an AI.

Does llms.txt actually work / do AI models use it?

Honestly: it is young and not yet universally honoured. Adoption is growing and several AI tools and crawlers reference it, but you should treat llms.txt as a low-effort, future-friendly bet rather than a guaranteed switch that turns on AI traffic. The bigger drivers of AI visibility are still being genuinely the right answer, clear and extractable content, and presence where models source. llms.txt is a sensible, cheap addition on top — not a substitute for the rest of GEO.

How do I create an llms.txt file?

Create a plain-text/markdown file named llms.txt and place it at your domain root. Start with an H1 of your site name, a short blockquote summary of what the site is, then markdown sections with bulleted links to your most important pages, each with a brief description of what it covers. Keep it curated — point models at your best, most useful content, not every URL. Optionally add an llms-full.txt with more detail. Then deploy it so it is reachable at yoursite.com/llms.txt. It takes minutes and needs no special tooling.

Is llms.txt different from robots.txt and sitemap.xml?

Yes. robots.txt tells crawlers what they may or may not access; sitemap.xml lists your URLs for search engines to crawl; llms.txt curates and describes your key content for large language models in human-readable markdown. They are complementary: keep robots.txt and sitemap.xml for classic SEO and add llms.txt for AI visibility. None of them guarantees anything on its own — they are signals that make it easier for the respective systems to do the right thing with your site.

Share X LinkedIn Facebook WhatsApp Reddit Email

Was this useful?