robots.txt decides, llms.txt suggests

I keep seeing posts sell llms.txt as "robots.txt for AI."

That claim is wrong. And it costs people hours on the wrong file.

What each file actually does

robots.txt answers one question: may you crawl this?

GPTBot, ClaudeBot, PerplexityBot, Google-Extended. The big ones publish that they read it. Same idea as Googlebot did in the late 90s. RFC 9309. Boring. Still the lever.

llms.txt answers a different question: if you're already here, what should you read first?

It's a Markdown reading list. No Allow. No Disallow. Google Search advocates have said flat out that Search does not use it as a ranking or crawl signal. One year of log studies keep showing the same thing: most consumer AI crawlers barely fetch it.

Where llms.txt earns its keep

Docs sites. Stripe, Vercel, Anthropic ship one so coding agents get a clean map. That's a real job. It is not "AI SEO."

What we shipped on MeetMila

MeetMila is our agentic application for surgeons. The launch order was:

1. robots.txt — public pages open, admin / login / API closed, AI agents listed by name
2. sitemap.xml — for discovery
3. llms.txt — a short product brief for assistants, not a ranking bet

If your goal is citation

If you want citation in ChatGPT or AI Overviews, start with robots rules per bot and a clean public graph. Treat llms.txt as optional courtesy after that.

I almost flipped that order myself. Glad I didn't.

Change my mind with a crawler log, not a vendor slide.