Ask around in any SEO Slack and you'll find two camps on llms.txt: people who added it to every client site in early 2025 because it seemed cheap and harmless, and people who never bothered because nobody could explain what it actually changed. Twenty-six months after the proposal first went up, there's finally data on who was right, and it's not close.

What llms.txt actually is

llms.txt is a plain markdown file you place at the root of your site, same spot as robots.txt, meant to give AI systems a condensed, navigable summary of your content instead of making them crawl and parse full HTML pages. Jeremy Howard, the Answer.AI co-founder known for his work on fast.ai and the Kaggle community, proposed it in September 2024. The format is simple on purpose: an H1 with your site or project name, an optional one-line blockquote summary, a short free-text section, then H2-headed lists of markdown links to the detailed pages you want an AI system to read, each with an optional one-line note. A second version of the spec shipped in August 2026, adding more formal rules for how those file lists should be structured.

Howard's framing at the time was about context windows and retrieval efficiency, not search rankings. Early large language models had limited context, and even as that's grown, feeding a model an entire rendered website is still wasteful compared to handing it a short index of what's available and letting it fetch only the pages it needs. That's a real engineering problem for anyone building a RAG pipeline or a coding agent that needs to read documentation on demand. The confusion that followed came from marketers extending that narrow, genuinely useful idea into a broader claim: that having this file would make a brand more likely to get cited in consumer AI search answers. Those are two different problems wearing the same file extension.

The pitch behind it is reasonable on its face. Web pages carry navigation chrome, ads, cookie banners and JavaScript that a language model doesn't need and has to work to strip out. A single, hand-curated markdown file in plain English removes that friction and points straight at the content that matters. llms.txt's own site claims thousands of sites now publish one, that Mintlify and other documentation platforms generate it automatically, and that Chrome's Lighthouse audits for its presence.

That adoption claim turns out to be the easy part. The harder question, whether anything on the other end actually reads the file, is where this gets interesting.

It's worth clearing up a common mix-up before going further: llms.txt is not robots.txt, and the two files do opposite jobs. robots.txt is a decades-old, universally-respected standard that tells crawlers what they're allowed to fetch, and every major search engine and AI crawler checks it before requesting a page. llms.txt doesn't ask permission or grant it, it's a suggested reading list, and nothing in the protocol requires any crawler or retrieval system to open it, let alone follow it. Confusing the two is how some site owners end up assuming llms.txt carries the same authority robots.txt does. It doesn't, and that gap between "universally checked" and "optional reading list nobody's confirmed reading" is the entire story of this article.

What Google says

Google's position has been unusually direct and consistent for something this new. Google's own developer guide on appearing in AI features states it plainly: "You don't need to create new machine readable files, AI text files, or markup to appear in these features," adding there's no special schema.org markup required either and no additional steps beyond standard technical SEO. That's not a hedge, it's Google telling site owners directly that llms.txt isn't part of how AI Overviews or AI Mode decide what to show.

Individual Search Relations team members have been even blunter in public settings. Gary Illyes said in July 2025 that Google doesn't support llms.txt. John Mueller has gone further than once: on Google's own Search Off the Record podcast in mid-2026, he argued that because the file is entirely self-reported by the site owner, a search system has no reason to trust it as a way of telling one site apart from another, and he's previously compared it directly to the long-dead keywords meta tag, a signal site owners controlled entirely and search engines stopped trusting decades ago for exactly that reason. He did carve out one narrow exception: if an automated agent is already on your site for some other reason, a short set of in-page instructions could function like a directory inside a store someone has already walked into. That's a real use case. It's also a much smaller claim than "add this file and AI search will find you."

As best we can confirm, Google has also stated directly that the file "will neither harm nor help visibility or rankings," which lines up with everything above: not penalized, not rewarded, not read for ranking purposes at all.

What other AI companies say

Here the picture is thinner, and that thinness is itself informative. As of early 2026, none of OpenAI, Google, Anthropic, Meta or Perplexity had publicly confirmed that their production answer systems actually read llms.txt files when deciding what to cite. Silence isn't the same as a denial, but after two years, an absence of any vendor saying "yes, our retrieval system checks for this file and it affects what we cite" is itself a data point.

Where it gets almost ironic: Anthropic, OpenAI and Google all publish an llms.txt file for their own developer documentation. Anthropic's developer docs serve one at platform.claude.com/llms.txt, a straightforward index of API documentation. That's a real, verifiable use of the format, it's just a completely different use case from "will this get my marketing site cited in a chatbot answer." Those companies are using llms.txt to help coding agents and developer tools navigate technical documentation efficiently, which is the use case Mueller specifically endorsed. None of it is evidence that the same companies' consumer answer engines read a brand's llms.txt file before deciding whether to mention that brand in a response.

What the data shows

Two independent studies, run almost a year apart with different methods, land on the same conclusion, and that convergence is the real story here.

Ahrefs analyzed server logs across 137,000 domains, finding that roughly 38,000 of them had a valid llms.txt file in place. Of those, 97% received zero requests at all in May 2026, the month the study covered. On the files that did get some traffic, AI retrieval bots (the category that would actually matter for citations) accounted for just 1% of requests. The biggest chunk of traffic to these files, 21%, came from SEO audit tools checking whether the file existed, not from anything reading it for content. Regular search crawlers like Googlebot made up another 13%. In other words: more of the traffic to the average llms.txt file comes from tools auditing its presence than from anything that might plausibly use its contents.

SE Ranking ran a larger, differently-designed study across roughly 300,000 domains, checking whether having an llms.txt file correlated with how often a domain got cited in AI-generated answers. About 10% of the sample had the file implemented, evenly distributed across low, medium and high-traffic sites, which already undercuts any theory that adoption tracks with sophistication or results. Using both simple statistical correlation and an XGBoost model, SE Ranking found no measurable relationship between having the file and citation frequency. Removing llms.txt from the model's inputs entirely actually improved its predictive accuracy, meaning the file wasn't just neutral, it was adding noise.

Coverage of these numbers kept surfacing through the back half of 2026. DesignRush ran a piece in late September under the headline "97% of llms.txt Files Go Unread," pointing at the same underlying pattern, and an October analysis on Security Boulevard made the meta-keywords comparison explicit in its headline, walking through Illyes' and Mueller's comments alongside the SE Ranking numbers and landing on the same verdict: a self-declared, unverified signal with no confirmed reader on the other end functions exactly like the keywords meta tag did in the 2000s; present on millions of pages, read by nobody who mattered, eventually ignored entirely.

None of this means llms.txt files are actively harmful to have sitting on your server. They're not slowing anything down, and Google has said directly they don't hurt rankings. The honest framing is that they're inert for the purpose most people installed them for.

There's a real security wrinkle too

Beyond the "does it help" question, 2026 surfaced a genuine downside that's worth flagging if you're running autonomous coding agents against documentation sites. Researchers demonstrated that AI agents, including Claude, Codex and other tools, could be tricked into executing arbitrary code inside corporate networks by having them follow instructions embedded in an llms.txt-style guidance file, treating a file that's supposed to be informational as if it were a trusted set of commands. That's a prompt-injection risk, not a Google-penalty risk, but it's a reason to treat any llms.txt file you don't control, especially ones your own coding agents might fetch from third-party docs, the same way you'd treat any other untrusted input: don't let an agent blindly execute instructions it finds in a text file just because the file has a recognized name. Our piece on whether to block AI crawlers covers the related question of what to let onto your site in the first place, worth reading alongside this if you're thinking about AI access policy broadly.

Where it can still genuinely help

Strip away the SEO framing and there's a real, narrow use case left standing: developer documentation consumed by coding agents. If you maintain API docs, SDK references or a technical knowledge base that tools like Cursor, Claude Code or Codex-style agents actually read while a developer is working, a clean llms.txt pointing at your markdown source files can save that agent time and tokens, which is precisely the "temporary crutch" framing Mueller used, not dismissively, just accurately scoped. This is also the exact use case Anthropic, OpenAI and Google are using it for on their own sites. If your business is selling developer tools, an SDK, or anything else with technical documentation that AI coding assistants might reference, building an llms.txt for that documentation is a reasonable use of an afternoon. If your business is a dentist's office, a SaaS marketing site, or an ecommerce store hoping to get name-checked in a ChatGPT answer about "best running shoes," the current evidence says skip it and spend the afternoon elsewhere. Our guide to generative engine optimization for beginners covers what actually does move AI citation rates, which is mostly the same fundamentals that work for Google: structured, specific, well-linked content that answer engines can parse without any special file pointing the way.

How to create one, if you still want to

If you've read all of the above and still want one, for the documentation use case or just because it costs little, the process is straightforward. Write a single markdown file named llms.txt, start with an H1 for your site or project name, add an optional one-line blockquote summary beneath it, then group links to your most important pages under H2 headings with a short note on each, following the format llmstxt.org documents. A minimal one for a documentation site looks like this:

# Acme API

> Acme is a REST API for widget inventory management.

## Docs

- [Quickstart](https://docs.acme.com/quickstart.md): Authentication and your first request
- [Endpoints reference](https://docs.acme.com/endpoints.md): Full list of routes and parameters
- [Rate limits](https://docs.acme.com/limits.md): Current quotas by plan

## Optional

- [Changelog](https://docs.acme.com/changelog.md): Recent API version changes

Notice the links point at .md versions of the pages where possible, that's deliberate, it saves the agent from having to strip HTML out of a rendered page. Place the file at your domain root (yoursite.com/llms.txt). If you run a documentation platform like Mintlify, Docusaurus or GitBook, check whether it already generates one for you, several do this automatically now. Keep it current the same way you'd keep a sitemap current, stale links in a file nobody's required to read are low-priority, but they're not free either.

The straight recommendation

For AI search visibility specifically, don't bother, and don't let anyone sell you an "llms.txt optimization service." The evidence from two separate studies, Google's own explicit statement that the file doesn't affect rankings or AI feature eligibility, and on-record comments from two members of Google's Search Relations team all point the same direction. Spend that time instead on the things our technical SEO checklist covers: crawlability, structured data that's actually consumed, content that answers the question clearly in the first few sentences. If you maintain developer-facing documentation that coding agents read, building one is still reasonable, just understand that's a developer-experience decision, not an SEO one, and keep it off your list of things to check when a client asks "why aren't we showing up in ChatGPT." For that question, the real levers are covered in our guide to getting cited by ChatGPT and Google AI Overviews.

FAQ

Does adding llms.txt hurt my SEO?

No. Google has stated it neither helps nor hurts rankings or AI feature eligibility. The risk isn't ranking penalty, it's opportunity cost: time spent on a file with no confirmed reader instead of on changes that do move the needle.

Is llms.txt an official Google or AI industry standard?

No. It's an independent proposal from Jeremy Howard and Answer.AI, published on llmstxt.org, not a standard adopted or endorsed by Google, OpenAI, Anthropic or any other major AI lab for ranking or citation purposes, even though several of those companies use it internally for their own developer docs.

If big AI companies publish llms.txt for their own docs, doesn't that mean it works?

It means it works for the specific use case they're using it for: helping a coding agent navigate technical documentation efficiently. It's not evidence that their consumer-facing answer products (ChatGPT, Gemini, Claude's web search) read a random business's llms.txt file before deciding what to cite in response to a shopper's question.

Should I remove llms.txt if I already have one?

There's no evidence it's actively harmful, so removing it isn't urgent. If keeping it current is eating any real time, the SE Ranking study's finding that excluding the file improved citation-prediction accuracy suggests it's safe to simply stop maintaining it and not look back.

What should I do instead to show up in AI answers?

Focus on the fundamentals that actually correlate with AI citations: clear, well-structured content that directly answers specific questions, solid technical SEO so crawlers and retrieval systems can access and parse your pages, and a real presence in the sources these systems already trust, reviews, forums, comparison sites. Our guide to answer engine optimization is the fuller version of this answer.

Didn't Bing or another engine confirm they use it?

Not that we could verify as of October 2026. If a vendor has made a clear public statement committing to reading llms.txt for ranking or citation purposes, that would change the calculus for that specific engine, but we didn't find one from any major player during this research. Treat any claim otherwise with the same skepticism this whole article is built on: ask for the specific source, not a marketing page repeating the proposal's own pitch.