Should You Ship llms.txt If Google Says No?

Short answer: yes, ship it. Google Search Central published an official guide on May 15 telling you llms.txt is not needed for AI Overviews or AI Mode. Eight days earlier, on May 7, Google Chrome shipped Lighthouse 13.3 with an llms.txt audit turned on by default in a new Agentic Browsing category. Two Google teams, two opposite positions, same week.

The cleanest read is that both teams are right about their own surface. Google AI search does not require llms.txt. The agentic web does. If you want the file to do work, ship it for the second use case and stop arguing about the first.

What Google Search actually said on May 15

On May 15, 2026, John Mueller published a new resource for optimizing for generative AI in Google Search on the Search Central Blog. The post links to a ~2,000-word guide now housed under a new "Generative AI fundamentals" section at developers.google.com/search/docs/fundamentals/ai-optimization-guide.

The guide is the first piece of official Google documentation scoped to AI Overviews and AI Mode. It legitimizes the category, then debunks four formatting myths the GEO vendor ecosystem has been selling. The first one names llms.txt directly.

The exact quote: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in generative AI search."

That is the cleanest possible Google statement that llms.txt is not a Google-AI-search ranking input. It also confirms Google does not require chunking, does not require a special AI writing style, and treats generative AI search optimization as "still SEO."

Most coverage stopped there. Search Engine Journal and Search Engine Land both led with the same headline. The GEO vendor ecosystem went quiet for ten days.

What Lighthouse 13.3 actually shipped on May 7

The story that broke the consensus surfaced on May 22, when Search Engine Land and Search Engine Journal noticed something the Google Chrome team had shipped two weeks earlier.

Lighthouse 13.3 introduced a new audit category called Agentic Browsing. The category moved from experimental to default config in this release. Inside it, the audit set includes a check for llms.txt.

The Chrome for Developers documentation lists what the audit actually checks:

• •Whether the site provides an llms.txt file
• •Whether the file has an H1 header
• •Whether the file has any content
• •Whether the file contains any links
• •Whether retrieval errors occur server-side

One critical scoring detail: Agentic Browsing is the only Lighthouse category that does not ship a weighted 0 to 100 score. The Lighthouse team chose actionable signals over a definitive ranking because "the standards for the agentic web are still emerging."

If the file is absent, the audit is marked Not Applicable. So llms.txt is optional today, but if it is present, it gets graded.

Why two Google teams disagree

This is not a contradiction so much as a product split.

Google Search Central owns AI Overviews and AI Mode visibility. Those surfaces are powered by Google's main ranking infrastructure. They do not fetch llms.txt, and they do not need to, because Google has been crawling the open web for 25 years.

Google Chrome Lighthouse owns developer tooling for site quality. Agentic Browsing is a new category meant to flag whether a site is ready for AI agents that visit it through a browser. Those agents are not Googlebot. They include Cursor, Claude Code, GitHub Copilot, Windsurf, ChatGPT operators, Perplexity Comet, and a long tail of MCP-driven workflows.

The same week as the Lighthouse 13.3 surfacing, Perplexity launched Comet for Enterprise with MDM deployment across macOS and Windows. OpenAI's Codex Goal Mode hit GA on May 23 with mobile preview and locked computer use. Both surfaces drive AI-agent traffic to brand sites. Neither is Google AI Overviews.

So the answer is not "Google contradicted itself." The answer is "Google has two product teams that own two different problems." The Search team is right that you do not need llms.txt to appear in AI Overviews. The Chrome team is right that the agentic web is now a measurable surface with its own readiness checklist.

What this means for your llms.txt decision

The old framing was simple. Ship llms.txt because some unspecified future AI may read it. Skip it because no major AI engine confirms reading it today.

That framing is dead. The new framing has two distinct audiences.

Google AI Overviews and AI Mode ask:

• •Is this page valuable, unique, and non-commodity?
• •Does the page answer the query directly?
• •Does the source pool already include this domain at depth?

AI agents in IDEs and AI browsers ask:

• •Where is this site's entry point for machine readers?
• •Which pages should an agent prioritize on a fresh visit?
• •Is there a clean orientation file that reduces the agent's setup cost?

The first set is unaffected by llms.txt. The second set is exactly what llms.txt was designed for. If you have buyers running queries in Cursor or Claude Code, you have an agent audience now. Most B2B SaaS teams do.

Five reasons to ship llms.txt anyway

These are the load-bearing arguments for shipping the file in May 2026. Each one alone justifies the half-day of work.

Reason 1: Lighthouse 13.3 grades it whether you ship it or not

The new Agentic Browsing audit ships in default config across every Lighthouse run from May 7 forward. That means every Google PageSpeed Insights report, every Chrome DevTools audit, every CI-pipeline Lighthouse check now surfaces llms.txt status. If your file is missing, the audit returns Not Applicable. If your file is present and broken, it surfaces an error your dev team can see.

Marketing-versus-engineering arguments about llms.txt are about to get a lot easier. Lighthouse is on the engineering side of that argument now.

Reason 2: IDE agents fetch it on first visit

Cursor, Claude Code, GitHub Copilot, and Windsurf all check for llms.txt when a user pastes your domain into a project context. The file is a single HTTP request. If it exists, the agent gets a curated map. If it does not, the agent falls back to crawling your site root and inferring structure from navigation.

The second path is more expensive for the agent and more noisy for your server. Both outcomes are worse for the user, who is the buyer you are trying to win.

Reason 3: Two Google teams disagree, and you should ship to the stricter one

When Google Search and Google Chrome disagree about a standard, the safer enterprise procurement posture is to follow the stricter requirement. Lighthouse is the stricter requirement. It now grades the file by default. Shipping it removes the question from your AI-readiness checklist.

The cost of compliance is low. The cost of being on the wrong side of a Google audit when buyers run Lighthouse on your site during evaluation is higher.

Reason 4: Perplexity Comet and Codex are agent-browser traffic now

Perplexity Comet shipped on iOS, Android, Mac, and Windows by May 21, with Comet for Enterprise launched May 22. OpenAI Codex Goal Mode shipped Appshots and mobile preview the same week. Both are agent-driven surfaces, and both behave more like Lighthouse-graded browsers than like Googlebot.

If you have decided your B2B SaaS buyers might use Comet, Codex, or Claude Code during evaluation, you have already decided agent-readiness matters. llms.txt is the cheapest line item on that readiness list.

Reason 5: The cost is half a day and never recurs

A reasonable llms.txt file is 40 to 80 lines of plain text at your domain root. It does not block rendering. It does not affect Core Web Vitals. It does not interact with your CMS. Ship it once and update quarterly when content priorities shift.

Compared to the cost of a single structured-data deployment, this is a rounding error.

Three reasons people skip llms.txt (and why each one is weaker than it looks)

These are the arguments you will hear in the next Slack thread on this topic. Each one has a real grain of truth, and each one fails the Lighthouse-13.3 test.

Skip reason 1: "Google said we don't need it"

Half right. Google Search said you do not need it for AI Overviews and AI Mode. Google Chrome added a default audit for it on the same product family. Both statements come from Mountain View. Picking one and ignoring the other is a choice.

If your buyers only ever discover you through Google AI Overviews and never through an AI browser or IDE agent, this argument holds. Most B2B SaaS teams have already lost that bet.

Skip reason 2: "No measurable citation lift from llms.txt"

True in the SEO-citation-study sense. Codersera's May 2026 review confirms there is no published evidence that llms.txt directly increases Google AI Overviews citations or ChatGPT mention rates. That is the wrong measurement.

The right measurement is agent-traffic quality, not citation share. An agent that lands on a clean orientation file completes the user's task faster. The buyer notices that. Citation lift is the wrong KPI for this file.

Skip reason 3: "It's not a standard"

Technically correct. llmstxt.org is a proposal, not an approved standard. No standards body has adopted it. Google itself just downgraded it in Search Central documentation.

Then Lighthouse made it a default audit. In product reality, "Google Chrome checks for it by default" is a stronger signal than "no W3C working group has approved it." Standards adoption follows tooling adoption, not the other way around.

How to ship a working llms.txt in under an hour

Five concrete steps. Most teams finish in a single afternoon.

Step 1: Pick the 8 to 15 pages an agent should know about

An llms.txt file is opinionated by design. List the pages that explain what your company does and where the deepest reference material lives. Resist the urge to dump your sitemap. A curated list of 10 strong pages beats 200 weak ones, the same logic Peec AI's 5.7M-data-point Listicle Rank Effect study found across AI engines.

Step 2: Write a 2-sentence company description at the top

The first H1 in your llms.txt file should name the company. The first paragraph should describe what the company does in two sentences a model can quote verbatim. This is how the agent decides what your site is about before it fetches a single linked page.

Step 3: Group your linked pages by purpose

Use H2 headings to separate categories. Typical groups: company overview, product or service pages, documentation, research and methodology, contact. Each link should be on its own line with a short one-line description. The Lighthouse audit specifically checks that the file contains links, so this step is also the test pass.

Step 4: Validate against the Lighthouse 13.3 audit

Run Chrome DevTools Lighthouse against your domain after the file is live. Open the Agentic Browsing category. Confirm llms.txt shows as Pass with no errors. If the audit returns an error, the most common causes are a 404 response, a missing H1, or a file that contains no links. Fix and re-run.

Step 5: Add the file to your quarterly content review

llms.txt is living infrastructure, not a deploy-and-forget asset. Review it every quarter. Remove links to pages you have retired. Add links to new flagship resources. The Lighthouse audit will continue grading it on every CI run, so out-of-date file content can quietly degrade your audit score over time.

What this looks like in practice

A short worked example. A B2B SaaS company that publishes a product, a docs site, three case studies, and a research blog. Their llms.txt at the domain root might look like this:

# Acme Analytics

Acme Analytics is a customer-data platform for product teams.
We help teams unify event data, build cohorts, and run experiments without a data engineer.

## Product
- https://acme.com/product
- https://acme.com/pricing
- https://acme.com/security

## Documentation
- https://acme.com/docs/quickstart
- https://acme.com/docs/sdks
- https://acme.com/docs/api-reference

## Research
- https://acme.com/blog/cohort-analysis-benchmark
- https://acme.com/blog/event-modeling-patterns

## Case studies
- https://acme.com/customers/series-b-fintech
- https://acme.com/customers/marketplace-scale

This file takes one hour to write, passes the Lighthouse 13.3 audit, gives Cursor and Claude Code a clean orientation, and changes nothing about how Google AI Overviews ranks the domain. That is the right set of outcomes for May 2026.

For the broader pattern on how agent-traffic strategy fits into your AEO program, see our Answer Engine Optimization complete guide and our guide on how to optimize for ChatGPT Search. If you want a deeper read on what llms.txt does and does not do at a content level, our llms.txt explainer covers the rest.

FAQ

Does Google use llms.txt for AI Overviews or AI Mode?

No. Google Search Central's May 15, 2026 generative AI optimization guide explicitly states you do not need llms.txt, markup, or Markdown files to appear in generative AI search. AI Overviews and AI Mode are powered by Google's main ranking infrastructure, which crawls the open web and does not require a separate machine-readable file.

Why does Google Chrome Lighthouse audit for llms.txt then?

Lighthouse 13.3 shipped a new Agentic Browsing category on May 7, 2026, with an llms.txt audit on by default. The audit is scoped to agentic-web readiness, which is a different problem than Google AI search visibility. The Lighthouse team is grading whether your site is ready for AI agents that visit through a browser, such as Cursor, Claude Code, GitHub Copilot, and Perplexity Comet.

Should I ship llms.txt if I do not care about AI agent traffic?

Probably still yes, because Lighthouse now grades it by default in every PageSpeed Insights and DevTools run. If the file is missing, the audit returns Not Applicable, which is fine. If you do ship it and a buyer runs Lighthouse on your site during evaluation, the audit returns a pass. The cost of shipping is a half-day of work and zero recurring maintenance.

Will llms.txt increase my AI citations?

There is no published evidence that llms.txt directly increases citation share in ChatGPT, Claude, Perplexity, Gemini, or Google AI Overviews. The file's value is agent-traffic quality, not citation lift. If you want a higher citation rate, the load-bearing inputs remain content quality, structured data, third-party listicle presence, and earned media. llms.txt is supporting infrastructure for agent workflows, not a ranking lever.

Does llms.txt replace robots.txt or sitemap.xml?

No. The three files serve different purposes. robots.txt controls crawler access at the protocol level. sitemap.xml lists every page that exists on your site. llms.txt is a curated orientation file that names which pages matter most and why, written for AI agents that benefit from a short reading list rather than a full sitemap dump.

The takeaway

Two Google product teams shipped opposite positions on llms.txt eight days apart. The Search team is right that the file is not a Google AI Overviews ranking input. The Chrome team is right that the agentic web is a graded surface now.

You do not have to pick a side. Ship the file for the audience that asked for it, which is the agent layer. Skip the debate about whether AI Overviews reads it, which it does not, and which does not matter, because that is the wrong product to optimize for here.

Half a day of work. Passes the Lighthouse audit. Reduces friction for Cursor, Claude Code, and Comet users on first visit. Updates quarterly. That is the right level of investment for the file in May 2026.