What Is an llms.txt File, and Does Your Site Need One?

A thread crossed your feed last week with a confident promise. Add one small text file to your website, it said, and start showing up when people ask ChatGPT for a recommendation. There were screenshots of a tidy little file, a few hundred approving replies, and somewhere in the comments an agency offering to install one for you, this week, for a fee. It reads like the cheat code everyone else missed.

The file is real. It is called llms.txt, it takes about five minutes to write, and shipping one is a reasonable thing to do. So this is not a debunking; it is a correction of proportion. On most of the sites getting sold this upgrade, the crawler that llms.txt exists to help never gets far enough to read it. The file is a welcome mat, and a lot of these houses have no door the crawler can open.

That gap, between the thing that got hyped and the thing that actually decides whether an AI assistant can quote you, is the whole post. The file is the last five percent. The other ninety-five sits underneath it, and nobody selling you a five-minute fix wants to talk about the ninety-five.

What is an llms.txt file?

An llms.txt file is a plain-text file you place at the root of your domain, at yoursite.com/llms.txt, that hands AI crawlers a curated map of your most important pages written in simple Markdown. It is the language-model cousin of robots.txt and sitemap.xml: where those tell a search engine what it may crawl, llms.txt tells a model what actually matters, in what order, and where the clean version of each thing lives. It is a proposal, not a ratified standard, and adoption among the major AI crawlers is still thin, which is the first reason to keep it in perspective.

The format is deliberately boring. A single top-level heading with your site or company name. A one-line summary in a blockquote. Then a few Markdown link lists grouped under plain headers: your core product pages, your best documentation, your key articles, each as a labeled link with a short description. Optionally a second file, llms-full.txt, that concatenates the actual text of those pages into one long document a model can read in a single pass without crawling anything.

That is the entire mechanism. It is not code, it does not run, and it changes nothing about how your site behaves for a human visitor. It is a signpost written in the one dialect a language model reads without effort: clean Markdown, no layout, no scripts, no noise. Which is exactly why it is worth doing, and exactly why it is nowhere near enough on its own.

Does your site need an llms.txt file?

Ship it, because it costs five minutes and cannot hurt you. And no, it is not the thing standing between you and ChatGPT traffic. If your pages are already clean, crawlable HTML with structured data underneath them, llms.txt is a sensible finishing move that helps a compliant crawler find your best material faster. If they are not, the file is a tidy label on a box the crawler cannot open, and writing it first means you have done the easy five percent and skipped the hard ninety-five.

So the honest answer depends on a question the thread never asked: what does an AI crawler actually see when it hits your homepage. There are two cases, and they end very differently.

In the first case, your site is built on real HTML. The words a customer reads sit in the page source, the headings are actual headings, and your pricing and services and articles are text a machine can lift without running anything. Here, llms.txt is pure upside. You are handing an already-capable reader a shortlist of your best pages so it does not have to guess. Add it and move on.

In the second case, your site is JavaScript soup, and that is where the five-minute fix quietly becomes decoration.

The file the crawler never reaches

Here is the failure the hype threads never screenshot. A business ships a beautiful llms.txt, points it at twelve carefully chosen pages, and nothing changes, because every one of those twelve pages renders as a blank shell until JavaScript runs, and the crawler that fetched them does not run JavaScript.

This is the default state of a large share of small-business sites, and it is not the owner's fault. Drag-and-drop builders and heavy client-rendered frameworks ship a nearly empty HTML document and then paint the real content in with scripts after the page loads. A human with a browser sees a finished page. A search crawler with a rendering budget sees it eventually, most of the time. An AI crawler fetching raw HTML on a tight budget frequently sees a gray rectangle where your headline should be. Your llms.txt dutifully points it at that gray rectangle.

We left an entire page-builder platform behind for exactly this class of problem, and wrote up why in why we left Wix. The short version: when the words that sell your business only exist after a script runs, you are betting your visibility on every machine in the chain choosing to run that script. Some do. Many, at the moment, do not. A website meant to work as a salesperson cannot sell to a reader who never sees the pitch.

The fix is ordinary infrastructure: server-rendered or static HTML, real semantic headings, your actual copy present in the page source, and structured data (schema markup) describing what each page is. When John D. Lantos, MD came to us on a legacy Wix site, the rebuild into a clean publishing platform served every essay as real text on his own domain, readable by a person, a search engine, and a model alike. The llms.txt would have been the last twenty minutes of that project, not the first.

An llms.txt file is a signpost. It is worth nothing pointing down a road the crawler cannot drive.

You do not have to take any of this on faith, and you should not. The fastest way to find out which case you are in is to look at your own site the way a crawler does. The free Pre-Flight Check fetches your pages and reads back what is actually there: whether your content sits in the HTML or hides behind scripts, whether your structured data exists, how fast the page renders, and where the machine-readable version falls short. Run it on your domain before you spend a minute writing an llms.txt, because it will tell you whether you are shipping a finishing move or a welcome mat to an empty house.

Will an llms.txt file get you into ChatGPT?

No, not on its own. Getting quoted by an AI assistant is mostly a function of three things: being crawlable, being structured, and being a recognized source on your subject. llms.txt is a thin convenience layer sitting on top of all three, not a substitute for any of them. It can help a well-behaved crawler find your best pages a little faster. It cannot make an unreadable page readable or an unknown business authoritative.

The mechanism that actually gets you cited is the one we broke down in SEO for AI search: answer real questions in clean, liftable prose, structure each page so a model can pull a self-contained answer, and earn enough corroboration across the web that the model treats you as a source worth repeating. The file does none of that work. It organizes the front door once the house is built.

It is worth separating this from a related idea that gets blended into the same threads. Becoming a recognized entity, a thing that Google's knowledge graph and the models trained on it actually know exists, is a different and slower project than dropping a text file at your root. We pulled those two apart in entity SEO: llms.txt is a file you write in an afternoon, and entity recognition is a reputation you accumulate over quarters. Confusing the two is how founders end up with a perfect llms.txt and a business no model has ever heard of.

So ship the file. It is cheap, it is harmless, and in a year the crawlers may honor it more than they do today, at which point you will be glad it was already there. Just do it in the right order. Write the llms.txt after your pages are real HTML a machine can read, after your schema describes what each one is, after your best answers exist as text instead of pixels painted on by a script. Do it before, and you have spent your five minutes decorating a wall.

If you are not sure which side of that line your site is on, start with the free Pre-Flight Check; it reads your site the way a crawler does and shows you the gap in a few minutes. If you would rather have the hard part built correctly, the clean HTML, the structured data, the pages a model can actually quote, with the little text file installed on top, book a call.