AEO fundamentals

What Is llms.txt, and Do You Need One?

llms.txt is a proposed plain-text file, written in Markdown, that gives AI agents a short guide to your website. It's cheap to publish and useful for documentation, but it isn't a way to rank in AI search: Google says you don't need it for AI Overviews or AI Mode.

3 November 2026IndexHalo Editorial Team6 min read

Contents

What it is

The llms.txt proposal was published by Jeremy Howard in September 2024 and updated in August 2026. Its reasoning is that web pages are built for people, with navigation and scripts around the information, while agents work best with concise text in one place. The file points them to it.

It's a proposal rather than an agreed standard. Its author reports that thousands of sites publish one, that documentation platforms generate them automatically, and that OpenAI, Anthropic and Google publish them for their developer documentation.

What goes in the file

The proposal sets out these parts, in this order:

A top-level heading with the site's name. This is the only required part.
A quoted summary with the key information needed to understand the rest of the file.
Optional detail in paragraphs or lists, with no headings.
Optional sections under second-level headings, each listing links to files.

EXAMPLE

# Example Company

> Example Company makes invoicing software for small agencies.

Plans are billed monthly. Prices are in US dollars.

## Docs

- [Getting started](https://example.com/start.md): Set up your first invoice

## Optional

- [Changelog](https://example.com/changelog.md)

You can see IndexHalo's own file for a real one.

How to check yours by hand

Open /llms.txt in a browser. It should load as plain text at that exact address, with no redirect to a web page.
Confirm the first line is a single # heading with your site's name.
Check that nothing comes before the summary, and that no headings appear between the summary and your first ## section.
Open every link. A dead link in the file is worse than no link.
Read it as a stranger would. If a sentence wouldn't help someone decide what to read, cut it.
Check your robots.txt too. If it blocks a crawler, this file won't change that. The AI crawler directory explains each bot.

Does it help with AI search?

Not in any way you should plan around. Google says plainly that you don't need new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode. Crawler access and clear pages come first. Being mentioned on the sites AI cites comes next, as our citation data shows.

So publish one if it's quick and you have something worth pointing to, particularly documentation. Just don't count it as your AI visibility work.

Primary references

Common questions

Does llms.txt help you appear in AI Overviews?

No. Google states that you don't need new machine-readable files or AI text files to appear in AI Overviews or AI Mode.

Is llms.txt the same as robots.txt?

No. robots.txt states what crawlers may fetch. llms.txt describes what is on your site and where to find it. A crawler you block in robots.txt stays blocked whatever llms.txt says.

Where does the file go?

At the root of your site, at /llms.txt. The proposal also allows one at a subpath, such as /docs/llms.txt, which covers the addresses under that path.

Who should bother with llms.txt?

Sites whose content agents read to get something done, such as developer documentation, get the clearest benefit. For a typical marketing site it takes little effort and does little harm, but don't expect it to change your visibility.