Both files sit in your website's root directory, both are plain text, and both are invisible to customers. That's where the similarity ends. robots.txt tells search engine crawlers which pages to ignore. llms.txt is a newer file that helps AI assistants understand what your business actually does and which pages matter most. If you only have one of them — or neither — you have a gap.
Here's what each file does, why the distinction matters for local businesses, and what to put in each one.
What robots.txt does
robots.txt has been around since 1994. Every search engine crawler (Googlebot, Bingbot, and dozens of others) checks this file before crawling your site. It tells them: you can index this, stay away from that.
A plumbing company might use robots.txt to block crawlers from a /admin login page or a /thank-you redirect that has no SEO value. Without those exclusions, Google might waste its crawl budget on pages you never wanted indexed in the first place.
What robots.txt does not do: it doesn't hide pages from humans, it doesn't improve your rankings, and — critically — it doesn't communicate anything meaningful to the AI assistants that are now answering customer questions. A blocked page just means the bot skips it. An AI system that already has cached data about your business won't consult robots.txt at all.
What llms.txt does
llms.txt is a proposal formalized in 2024, and adoption is growing fast. The idea is simple: put a structured plain-text file at yourdomain.com/llms.txt that gives AI systems a concise, authoritative summary of who you are, what you offer, where you're located, and which pages contain the most reliable information.
Think of it as a briefing document for AI. When a tool like ChatGPT, Perplexity, or Claude crawls your site to build its knowledge base, a well-written llms.txt makes it far more likely the model lands on accurate facts rather than assembling a patchwork from old cached pages, outdated press mentions, or a competitor's site that happens to mention your name.
A dental practice in a mid-sized city might have a llms.txt that reads something like:
# Riverside Family Dentistry
Riverside Family Dentistry is a general and cosmetic dental practice
located at 412 Oak Street, Springfield, IL 62701.
Hours: Mon–Fri 8am–5pm, Sat 9am–1pm. Closed Sunday.
Accepts: Delta Dental, Cigna, Blue Cross. Emergency slots available same-day.
## Key pages
- /services: Full list of procedures including whitening, implants, and Invisalign
- /new-patients: Insurance, forms, and first-visit FAQ
- /contact: Phone, directions, and online booking
That 100-word snippet does more to correct AI hallucinations than most technical SEO work. Understanding why AI gets your business info wrong usually starts here: the model never had a clean source to pull from.
The difference in one sentence
robots.txt manages which bots can crawl which pages. llms.txt tells AI systems what's true about your business.
They solve different problems. A bakery that blocks its /cart pages from crawlers is doing good technical hygiene. But if that same bakery's llms.txt clearly states it's a nut-free facility, it's far more likely an AI will report that correctly when a parent asks "which bakeries near me are safe for nut allergies?"
Why local businesses need both
Search engine rankings still matter — a lot. robots.txt hygiene keeps your crawl budget clean, prevents duplicate content issues, and stops Google from indexing pages that dilute your relevance signals. Don't neglect it just because AI is the new focus.
But AI assistants are increasingly the first place customers go for local recommendations. Someone asking ChatGPT "who's the best HVAC company in [city]" or "does [business name] offer financing?" is not looking at your title tags or meta descriptions. They're getting a synthesized answer from whatever data the model has ingested. If that data is thin, contradictory, or months out of date, the answer will be wrong.
Our guide to llms.txt for local businesses covers the full format, but the baseline is short: name, address, hours, services, accepted payment methods or insurance, and links to your three or four most important pages. That's enough to give an AI system a reliable anchor.
Common mistakes to avoid
Blocking AI crawlers in robots.txt by accident. Some site owners add broad User-agent: * disallow rules without realizing they're also blocking AI crawlers like GPTBot, ClaudeBot, and PerplexityBot. If your robots.txt is too restrictive, AI tools simply won't have current data about you — and they'll fall back on whatever stale information they can find elsewhere. Check your robots.txt and make sure you're not accidentally locking out the crawlers that feed AI knowledge bases.
Writing llms.txt once and forgetting it. If your hours change, you add a service, or you move locations, your llms.txt needs to match. An AI that reads an outdated file will confidently give customers wrong information. Build a reminder into your quarterly business review.
Skipping llms.txt because the site is "small." A one-location hair salon with eight pages and no blog has the most to gain from a concise llms.txt — there's less on the site for an AI to read, so the briefing document carries more weight.
Where to start
Run a free scan to see how AI assistants currently describe your business. If the grade is a C or below, the two most common fixes are a properly formatted llms.txt and a check that your robots.txt isn't accidentally blocking AI crawlers. Both take under an hour and cost nothing to create. Start with the scan so you know exactly what you're fixing before you touch any files.