llms.txt: The Complete Guide for SEO

What Is llms.txt and Why Is Everyone Talking About It?

llms.txt is a plain text file, written in markdown, that sits at the root of a website, for example example.com/llms.txt. It lists a site’s most useful pages in a short, structured index so that AI systems can find them without crawling an entire domain. Jeremy Howard, co-founder of Answer.AI, proposed the format in 2024 as a lightweight way to hand large language models and AI agents a curated map of a site’s content.

The file does not control crawling the way robots.txt does, and it does not directly influence search rankings the way a sitemap can support indexing. It is closer to a curated table of contents written for machines rather than for search engine crawlers. That distinction matters more than most SEO content admits, and it shapes almost everything covered in this guide.

How Does an llms.txt File Actually Work?

A standard llms.txt file opens with an H1 title naming the site, followed by a short blockquote summary of what the business does. Below that, links to important pages are grouped under headings such as Documentation, Guides, or Products, each with a one line description. When a compatible crawler requests the file, it receives this condensed list instead of parsing full HTML pages loaded with navigation bars, scripts, and ad markup.

The Companion Format: llms-full.txt

A related file, llms-full.txt, goes a step further by embedding the complete markdown text of every linked page inside a single document, rather than just linking out to it. Adoption of llms-full.txt is smaller than llms.txt itself but has grown quickly off a tiny base, largely among developer tooling and documentation sites where the entire knowledge base benefits from being available in one clean pull.

Does Google Use llms.txt for Search Rankings?

The direct answer is no. Google has said repeatedly that it does not use llms.txt for crawling, indexing, or ranking, and it has no plans to change that position. Gary Illyes of Google Search Relations confirmed at Google Search Central Live that Google does not support the format. John Mueller has separately compared llms.txt to the old keywords meta tag, a signal that search engines stopped trusting decades ago because site owners controlled it directly and could use it to describe a page more favorably than the page actually deserved.

In December 2025, an llms.txt file briefly appeared on one of Google’s own developer documentation properties, and SEO forums treated it as a quiet endorsement. Google clarified that an internal content system had added the file automatically and that no search or AI feature was consuming it. It was removed within hours. Separately, Google’s mid-2026 guidance on generative AI features includes a section that directly tells site owners a machine readable file like llms.txt is not required to appear in AI Overviews or similar surfaces.

This creates genuine friction with Chrome’s Lighthouse audit tool, which does flag the absence of an llms.txt file as part of its AI readiness checks and describes it as an emerging convention. Both statements are technically accurate at once. The Search team and the Chrome team maintain separate documentation, and the gap between the two has fueled a lot of confusion that outlives the actual facts.

Which AI Platforms Actually Read llms.txt in 2026?

No major AI platform, including OpenAI, Anthropic, Google, or Meta, has formally committed to reading llms.txt for search or citation purposes. A large scale analysis by Ahrefs covering roughly 137,000 domains found that 28 percent of the sites studied had published an llms.txt file, yet 97 percent of those files recorded zero requests from any AI crawler over the study period. The traffic that does hit the file skews heavily toward SEO audit tools checking whether the file exists, rather than AI systems actually retrieving it.

Where llms.txt shows genuine utility is narrower than the SEO industry’s framing suggests. Developer focused AI tools such as coding assistants and agent frameworks are more likely to reference a documentation site’s llms.txt file when a developer is working directly inside that tool, since the original proposal was built for exactly that use case. For a typical business website chasing visibility inside ChatGPT, Perplexity, or Google AI Overviews, the file currently does very little.

How Many Websites Have Actually Implemented llms.txt?

Adoption has grown quickly in relative terms while staying small in absolute terms. Tracking firm Originality.ai recorded 4,088 llms.txt files across a sample of more than three million websites in June 2025, a number that reached 36,120 by May 2026, an 8.8 times increase in twelve months. A separate study by SE Ranking covering 300,000 domains put overall adoption at roughly 10 percent. Rankability’s monthly crawl of the world’s top 1,000 websites found 8.7 percent adoption as of June 2026, with the technology sector leading every other category at 36.4 percent and government websites sitting at zero.

The growth is real, but so is the ceiling. An analysis covering more than 300,000 domains found no measurable correlation between having an llms.txt file and how often a domain gets cited in AI generated answers. A separate machine learning model built by SE Ranking actually improved its citation prediction accuracy after the llms.txt variable was removed entirely, meaning the file added noise rather than signal to the model.

llms.txt vs Robots.txt vs Sitemap.xml: What Is the Difference?

These three files get grouped together constantly because they all live at the site root, but they solve unrelated problems. The table below lays out the practical differences.

File	Primary Purpose	Who Reads It	Affects Rankings
robots.txt	Controls what crawlers may access	Search engines, most AI crawlers	Indirectly, via crawl access
sitemap.xml	Lists pages for discovery and indexing	Search engines	Indirectly, via indexing efficiency
llms.txt	Curated markdown index for AI context	Developer tools, some AI agents; not confirmed for major AI search	No confirmed effect

How Do You Create an llms.txt File? The C.I.T.E. Framework

Because the format has no enforcement and no single authority, quality depends entirely on the process behind it. The C.I.T.E. framework below is a simple way to keep the file useful rather than turning it into a dumped list of every URL on the domain.

Curate Your Highest Value Pages

Select the pages that best represent what the business does and does best, typically ten to thirty entries. Cornerstone guides, pricing or service pages, and genuinely authoritative resources belong here. Thin category pages and duplicate landing pages do not.

Isolate the Clutter

Strip each linked description down to a single clean sentence. The value of markdown over HTML is that navigation, ads, and boilerplate disappear, so the description should follow that same discipline rather than repeating meta description copy.

Tier by Priority

Group links under clear category headings, such as Guides, Product Pages, or Policies, in the order that matters most to a visitor or an agent trying to orient itself quickly.

Evaluate and Refresh Regularly

Treat the file as a living document. A quarterly review to remove outdated links and add new cornerstone content keeps it accurate, which matters if adoption among AI systems eventually catches up to the current hype.

What Should and Should Not Go Into an llms.txt File?

• Include a short, factual summary of the business in the opening blockquote
• Include links to genuinely authoritative guides, documentation, or product pages
• Include clear category headings that group related links together
• Avoid listing every page on the domain, including thin or duplicate content
• Avoid marketing language or keyword stuffed descriptions inside the file
• Avoid treating the file as a substitute for a proper sitemap or robots.txt

Is llms.txt Worth Implementing for Your Website?

For most businesses, the honest answer is that it costs very little and delivers very little, at least today. If a content management system or SEO plugin can generate the file automatically, there is minimal downside to switching it on. If it requires dedicated developer time to build and maintain by hand, that time is almost always better spent on the fundamentals that AI systems and search engines both reward: clear entity definitions, well-structured headings, original data, and content that answers a question more completely than the page ranking above it.

Across recent content production work covering AI search topics, one pattern shows up consistently: the sites getting cited inside AI generated answers are winning on content depth, clean HTML structure, and topical authority rather than on the presence of a root level text file. That observation lines up with what the larger adoption studies keep finding. Treat llms.txt as a small piece of forward looking infrastructure for the agentic web, not as a shortcut around the harder work of earning citations.

What Are the Most Common Mistakes When Implementing llms.txt?

• Auto-generating the file through a plugin and never reviewing what it actually lists
• Listing hundreds of URLs instead of a curated, prioritized set
• Letting the file go stale after a site redesign or content pruning project
• Assuming the file replaces schema markup, internal linking, or crawlable HTML
• Marketing llms.txt internally as a guaranteed ranking or citation booster

Frequently Asked Questions

Does llms.txt improve Google rankings?

No. Google has confirmed on multiple occasions that llms.txt plays no role in crawling, indexing, or ranking. Google Search Relations has directly compared it to the discontinued keywords meta tag, a self-declared signal the company stopped trusting years ago because site owners controlled it without any verification.

Does ChatGPT or Perplexity read llms.txt?

There is no public confirmation that ChatGPT, Perplexity, Gemini, or Claude use llms.txt for search or citation purposes. Independent analysis of AI crawler traffic found that the overwhelming majority of published llms.txt files receive zero requests from AI systems, with most traffic coming from SEO audit tools instead.

Is llms.txt the same as robots.txt?

No. Robots.txt is an officially recognized standard that controls which pages crawlers may access. llms.txt is an unofficial, community proposed markdown index with no enforcement mechanism, and it cannot block or permit anything.

How many websites currently have an llms.txt file?

Estimates vary by study, but adoption sits in the single digit to low double digit percentage range depending on the sample. Rankability found 8.7 percent adoption among the top 1,000 global websites in June 2026, while a broader SE Ranking study of 300,000 domains found roughly 10 percent.

What should an llms.txt file include?

A short factual summary of the site, followed by a curated, categorized list of the most valuable pages, each with a single clean description. It should exclude thin pages, duplicate content, and promotional language, and should be reviewed regularly rather than generated once and forgotten.

Is llms.txt worth the effort for a small business site?

If a plugin or content management system can generate it automatically, there is little reason not to enable it. If it requires meaningful developer time, that time is generally better invested in content depth, structured data, and technical SEO fundamentals, since those are the signals with confirmed influence on both search and AI visibility.

Will llms.txt matter more in the future?

Possibly, particularly as autonomous AI agents that navigate websites to complete tasks become more common. The current evidence supports llms.txt as early infrastructure for that agentic future rather than as a search or citation ranking factor today

A Note From Stay Digital Marketers

Questions like whether a new technical file will move the needle come up constantly in link building and content strategy work, and llms.txt is a useful case study in separating genuine signal from SEO speculation. Stay Digital Marketers works with brands on the fundamentals that consistently hold up across both traditional search and AI driven discovery, including guest posting, press release distribution, SaaS backlinks, niche edits, and Google Knowledge Panel creation, alongside broader SEO strategy that does not depend on unverified shortcuts.