GEO & AI Search

llms.txt: what it does — and what it does not

llms.txt is a proposal for a Markdown text file in the root of your website that gives language models a concise overview of your most important pages. It is not a recognised standard, it does not control access and it is not a shortcut to being mentioned in AI answers.

At a glance

What is llms.txt? The short answer

An llms.txt file sits at jouwdomein.be/llms.txt and contains, in plain readable text, the name of your website, a short description and a list of links to your core pages. The idea behind it: a language model has limited space and wastes time on menus, scripts and markup. A compact overview helps it reach the right content faster.

The file is a proposal from 2024, not a standard that AI systems have to follow. It does not tell a crawler what it may or may not fetch: robots.txt does that. And it does not make an unclear website clear.

An llms.txt costs little to add and does no harm. Just do not expect it to deliver visibility that your website itself does not earn.

llms.txt – example of the structure: name, short description and a list of links

Origin

Where does llms.txt come from?

The proposal was published in September 2024 by Jeremy Howard of Answer.AI. The format is described at llmstxt.org. The starting point was technical documentation: anyone who has an AI assistant help them with programming wants that assistant to find the right manual quickly, without having to work through the entire website.

Alongside llms.txt there is a variant, llms-full.txt, which contains not just links but the full content of the most important pages in a single file. That variant is mainly used for software documentation.

The proposal has not been formalised by a standards body. That is a fundamental difference from robots.txt, which is a formal internet standard. Calling llms.txt ‘the new robots.txt’ therefore makes it sound stronger than it is.

Example

What does an llms.txt file look like?

The format is Markdown. Only the title is required; the rest is recommended.

The details in the example are fictitious.

Example: example.com/llms.txt
# Voorbeeldbedrijf

> Voorbeeldbedrijf bouwt en onderhoudt websites voor KMO's in België.

## Diensten

- [Website laten maken](https://example.com/website-laten-maken/): aanpak, prijzen en werkwijze
- [Onderhoud](https://example.com/onderhoud/): updates, back-ups en beveiliging

## Over ons

- [Over Voorbeeldbedrijf](https://example.com/over/): wie we zijn en voor wie we werken

## Optional

- [Blog](https://example.com/blog/): achtergrondartikels
  • A title with the name of the website or project.
  • A blockquote with a summary in one or two sentences.
  • Sections with links, each with a short explanation.
  • An ‘Optional’ section for content a model may skip if space is limited.

Usefulness

What does llms.txt do?

The file has a modest but real use, especially in these situations.

  • Providing an overview Tools and assistants that deliberately fetch the file get the structure of your website in one go.
  • Making documentation accessible Software, APIs and manuals are exactly what the proposal was intended for.
  • Pasting it into a conversation yourself You can give the file to an AI assistant when you ask questions about your own website or product.
  • Naming your core pages Writing it forces you to choose which pages really matter. That exercise is useful in itself.

Scope

What does llms.txt not do?

  • It does not control access. You allow or block crawlers in robots.txt, not here. See robots.txt and AI crawlers.
  • It does not secure a place in answers. As far as is publicly documented (November 2026), none of the major providers says that its search or answer system uses the file to compile answers.
  • It is not a requirement for Google. Google states in its documentation on AI features and your website that no separate files for AI are needed to appear in AI Overviews or AI Mode.
  • It is not a substitute for good pages. A list of links to vague or slow pages is still a list of links to vague or slow pages.
  • It is not a hidden channel. Do not put anything in it that is not also visible on your website. Showing machines different content from what people see is deception.

So the file is a peripheral aid, not a foundation. That foundation remains a website that is technically accessible and clear in its content.

Comparison

llms.txt, robots.txt and sitemap.xml: what is the difference?

The three files sit in the same place, but they do different things. They do not replace one another.

llms.txt, robots.txt and sitemap.xml side by side
Characteristicllms.txtrobots.txtsitemap.xml
GoalA readable overview of your core contentStating which crawlers may fetch which pathsListing all indexable URLs
FormatMarkdownPlain text with rules per user agentXML
StatusProposal from 2024Formal internet standardWidely supported protocol
Intended forLanguage models and tools that fetch itAll crawlersSearch engines
Effect on visibilityNot demonstratedMajor: one mistake can shut off your websiteHelps pages get discovered

If you have to choose where to spend your time, start with robots.txt and your sitemap. Those are what your visibility really depends on.

Deciding

Do you need llms.txt?

For most business websites, the answer is: you can, but you do not have to. Here is how to weigh it up.

Worthwhile when

  • you offer software, an API or extensive documentation
  • your website has a lot of content and the essentials are hard to find
  • the file is updated automatically along with your website
  • the foundations of your website are already in order

Of little use when

  • you see it as a replacement for technical SEO
  • your most important pages are themselves still unclear
  • nobody will maintain the file
  • you expect it to get you mentioned in ChatGPT

An outdated llms.txt with dead links is worse than no file at all. Only add it if it evolves along with your website.

Get started

How do you create an llms.txt file?

If you still want to add one, it is a small job. Some SEO plugins for WordPress can generate the file for you; you can also write it by hand.

01

Choose your core pages

The pages that explain who you are, what you offer and how you work. Not your entire sitemap.

02

Write the summary

One or two sentences that make sense even without the rest of your website.

03

Group the links

By section, with a short note for each link on what the page answers.

04

Put it in the root directory

The file must be reachable at /llms.txt and served as text.

05

Keep it up to date

Update it whenever pages disappear, move or are added.

Priorities

What does help visibility in AI answers?

GEO (Generative Engine Optimization) relies on work that takes longer than a single file, but that makes a demonstrable difference to how systems read your website.

  • Accessible pages Content that is in the HTML and that crawlers are allowed to fetch.
  • A recognisable identity The same name, description and details everywhere. Read Entity SEO.
  • Answers up front A question as the heading, the answer in the first sentences.
  • Accurate structured data Markup that describes what is visible on the page.

What GEO involves exactly is explained in What is GEO? You can read how I approach it as a service on GEO & AI SEO.

FAQ

Frequently asked questions about llms.txt

LLM stands for large language model, the kind of language model behind systems such as ChatGPT, Gemini and Claude. An llms.txt is a text file intended for such models that summarises the most important content of a website.

You do not have to. For documentation and technical products, it is useful. For an ordinary business website, the effect has not been demonstrated: it does no harm, but your time pays off more when spent on clear pages, a correct robots.txt and an up-to-date sitemap.

No. Google says that no separate files are needed for its AI features, and the file plays no role in the regular search results either.

Type the domain into your browser, followed by /llms.txt. If you see a text file, it is there. If you get an error page, it is not.

llms.txt contains an overview with links. llms-full.txt contains the full text of the most important pages in a single file, so that a model can read everything in one go. The second form is mainly common for software documentation.

As an aid for documentation, yes. As a way to get into AI answers, it has never been proven. The proposal may still evolve, so it pays to follow the official documentation of the AI providers rather than relying on promises made by tools.

More on this topic

Further reading

The related service

GEO & AI SEO as a service

Visibility in AI answers takes more than one file. The service page explains how I approach technology, entity and content together.

Dries Nijs, Organic Web Specialist and author of this article

About the author

Dries Nijs, Organic Web Specialist

I am Dries Nijs, a developer and SEO consultant from Maasmechelen. I follow new proposals such as llms.txt closely and weigh them against what the providers themselves document. MadeByDries is the personal brand under which I work.

Not sure where your time will pay off most?

Tell me about your website. Then we will look at what needs to come first: the technology, the content or your entity.

Discuss your project

Tell me what you want to build or improve

You do not need a technical brief yet. Tell me where you are now and what you want to achieve, and I will help you find the right approach.

Choose at least one topic. You can select more than one.

I only use your details to respond to your enquiry.

Prefer to email me directly?

Location
Maasmechelen, Belgium