Does llms.txt work? What the data shows

A founder is checking a product page before a buyer call. Someone on the team added an llms.txt file last quarter and nobody has looked at it since. She types a blunt question into the search box: does llms.txt work?

The short answer from our data is plain. The file is cheap to add, 66 of 121 audited sites already have one, and our citation records show no identified llms.txt source. That is evidence of adoption, not evidence that the file improves visibility. Most of the 66 sites with a file will not know from this data whether it changed anything.

Does llms.txt work for your citations?

The proposal has a clear job. The llms.txt specification says, “We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content.” It describes a file that can sit at the site root or under a path, then point an agent towards useful pages. That makes it a navigation aid, rather than a replacement for the pages it links to.

The proposed workflow is also clear: “Agents are expected to view or search llms.txt to find the information they need, then follow the relevant links.” That could help an agent navigate a large documentation site. It does not, by itself, make a page authoritative, relevant to a buyer’s question, or likely to be selected as a citation.

The llms.txt check passed on 66 of 121 sites and warned on 55, according to the CiteGraph AI readiness checks. A warning is the state for sites that simply have no file, so the check does not fail a site outright for lacking one. The check records whether the file is present for that audit, not whether a passing site receives more citations.

Across 45,309 citations in 7,260 AI answers, the citation-by-kind breakdown does not identify llms.txt as a cited source. That is an inference from the recorded source categories, not a separately logged zero-count experiment. The citation is normally a page the assistant used as a source, not the navigation file that may have helped it find that page. A signpost is not the shop.

The chart below shows the pass, warn and fail rates for the fifteen readiness checks, with llms.txt among the checks measured across 121 sites.

The result answers “llms txt is it used” more narrowly than some people expect. We can see that many audited sites publish the file. We cannot tell from these records whether an agent read one before citing a linked page.

What does llms.txt do for your site?

The spec is more specific than the name suggests. It says that “llms.txt markdown is human and LLM readable, but is also in a precise format allowing fixed processing methods,” including parsers and regular expressions. The required structure is modest: “An H1 with the name of the project or site. This is the only required section.” File lists then use Markdown links, with optional notes, so the format gives an agent both a page and some context about it.

The file is meant to help an agent choose where to look. The specification says, “A file covers the URLs under its path, and where more than one file applies, agents should use the most specific one.” It also says the format is designed to work alongside existing web standards: “llms.txt is designed to coexist with current web standards” and “robots.txt and llms.txt have different purposes.”

That difference gives you a sensible way to think about where llms.txt fits among your other technical signals. robots.txt deals with whether automated tools may access parts of your site. A sitemap exposes URLs. Normal pages contain the facts an assistant can quote, while llms.txt is a curated map between those things. It is not a ranking signal in the evidence we have, and the specification does not turn it into one.

The spec says files are “used most heavily for software documentation, where coding agents follow them to find API references and tutorials.” That is a narrower use case than a buyer asking for a shortlist of products. A documentation agent may benefit from a clean index even when a recommendation system still chooses sources based on the pages, sites and question context it can retrieve.

There is little downside to adding the file if you can keep it accurate. Use it to point to stable documentation, pricing, reference material or other pages that already explain the product, but do not treat it as a substitute for those pages. If the linked material is vague, inaccessible or written for internal comfort rather than buyer understanding, the file simply points more efficiently at the same problem.

Which signals matter more than llms.txt for your site?

The readiness audit gives useful context. In the same 121 sites, AI crawlers were allowed in robots.txt on 116 sites, a sitemap was published on 100, and one clear H1 was present on 101. Those are basic conditions, but they affect whether an assistant can reach and interpret the page at all. They are also easier to inspect than an invisible retrieval decision.

Other checks show larger gaps. A plain statement of what the product is, who it is for, what it costs and what it replaces passed on only 20 sites. Questions were answered on the page on 56 sites. Comparison or alternatives pages passed on 54. Those are closer to the work an assistant needs to answer a buyer’s question than a file that lists where the answers might live.

The citation mix points to the same practical priority. Listicles and roundups account for 67.5% of recorded citations, while competitor product pages account for 24.4%. The product’s own site accounts for 1.5%. Even before llms.txt enters the picture, only 5 of 121 sites passed our check for whether their own pages get cited at all. Assistants are often citing pages built to compare, filter or explain products, rather than a site index.

The chart below shows the share of recorded AI citations by page type, including 67.5% for listicles and roundups, 24.4% for competitor product pages and 1.5% for the product’s own site.

That does not make your own site irrelevant. It means the page carrying the answer is usually more important than the file describing the site. Our weekly research on who owns AI answers shows the same practical problem from another angle: citations concentrate in page types that compare, filter or explain products for a buyer.

If you have one hour, spend it in a dull but useful order: get the product statement, the price and the buyer-question answers right, then publish comparisons where the product genuinely differs, and check that the important pages are crawlable and readable without JavaScript. Add llms.txt once those foundations are in place, not instead of them. There is no prize for producing a neat index of unclear pages.

What can you not tell from your llms.txt data?

We measure samples, not the whole web. We print the sample size, and we do not know what we have not measured. The readiness data covers 121 audited sites. The citation data covers 60 scans of 41 sites in 44 categories, with 442 distinct buyer questions, so it is broad enough to show patterns but not large enough to settle every use case.

We also cannot tell whether an individual assistant read a particular llms.txt file and then used a linked page. The citation record shows the cited page, not every step in retrieval. If a file helped an agent discover a page, that assistance could remain invisible in the final answer.

We cannot say that llms.txt never helps. Documentation platforms and coding agents may use it in ways our buyer-question scans do not expose. The specification describes software documentation as the main use case, and our citation sample is not a controlled experiment comparing identical sites with and without the file.

We can say what the available evidence does show. Access, clear product facts, answers to buyer questions and comparison content are measurable parts of the citation path. The role of llms.txt, if it helped an agent reach one of those pages, is not visible in the final citation record.

Our own free AI visibility checker treats llms.txt as one readiness signal among several, not as a verdict. That is the sensible position while adoption is visible and measurable citation impact is not.

The caveat that matters most is simple: these records do not show that adding llms.txt improves citations, and they cannot show every retrieval step before a citation appears.