An llms.txt file is a markdown document placed at your domain root that tells AI systems which pages matter most. Implementing one correctly takes about an hour, but most teams get it wrong by either including too much or structuring it poorly.
This guide walks through the full implementation process: auditing your content, writing the file, choosing what to include for your specific site type, validating the output, and measuring whether it makes any difference. You'll find ready-to-use templates for SaaS companies, e-commerce stores, publishers, and agencies.
What Is llms.txt and Why Does It Matter for AI Search?
An llms.txt file is a standardized markdown document hosted at `yourdomain.com/llms.txt`. It gives AI systems a curated map of your most important content: documentation, pricing, policies, product pages, and anything else you want large language models to prioritize when generating answers about your brand.
Think of it as the third layer in your site's machine-readable stack:
- robots.txt tells crawlers what they can and cannot access
- sitemap.xml lists URLs for search engine indexing
- llms.txt tells AI systems which content is most important and where to find clean versions of it
The concept was introduced by Jeremy Howard (creator of fast.ai) and has gained traction among developer-focused companies, SaaS platforms, and documentation-heavy sites. As of mid-2026, the llmstxt.org directory lists hundreds of companies that have published llms.txt files, from startups to enterprises.
If you want a deeper dive into what llms.txt is and whether it currently impacts AI visibility, read our explainer on llms.txt and AI search visibility. This guide focuses purely on the technical implementation.
Does Google Treat llms.txt Differently?
No. Google addressed this directly in its May 2026 AI search optimization guidance: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in generative AI search." Google may discover and crawl many file types beyond HTML, but that does not mean llms.txt receives special treatment.
This is worth understanding before you invest time. The file will not hurt your site, and it may become more relevant as AI retrieval systems evolve. But right now, the primary value is organizational: it forces you to identify and curate your most important content, which is a useful exercise regardless of whether crawlers read the file.
How to Implement llms.txt in 5 Steps
Here is the complete implementation process. Most teams can finish this in under two hours.
Step 1: Audit Your High-Value Content
Before writing a single line of markdown, identify the 10-30 pages that best represent your brand. Ask yourself: "If an AI system could only read 20 pages from my site, which ones would I want it to see?"
Pages to prioritize:
- Core product or service pages
- Pricing and plan details
- API documentation and integration guides
- Key policies (privacy, terms, returns, SLAs)
- Onboarding or getting-started guides
- Your most authoritative blog posts or research
- Case studies with specific data points
Pages to exclude:
- Navigation-heavy landing pages with little substance
- Deprecated documentation or legacy product pages
- Marketing pages with mostly visual content and minimal text
- Internal tools or admin pages
- Duplicate content (pick the canonical version)
AI Overviews now appear in 25.11% of Google searches, which means a growing share of your audience encounters AI-generated answers before they ever reach your site. The pages you include in llms.txt should be the ones you want AI systems to reference when generating those answers.
Step 2: Create Clean Markdown Versions
llms.txt works best when it links to content that AI systems can parse without fighting through navigation bars, cookie banners, JavaScript widgets, and layout code. The ideal target is a clean markdown (.md) file for each important page.
Options for generating markdown versions:
- Manual conversion: Write .md files for your top 10-15 pages. This gives you the most control over what AI systems see.
- Automated mirroring: Tools like Mintlify, Docusaurus, or custom scripts can generate markdown from your existing HTML docs.
- CMS-native markdown: If your CMS stores content in markdown (Ghost, Hugo, Astro, many headless CMS platforms), you may already have .md versions available.
- llms-full.txt alternative: Instead of individual .md files, some companies create a single `llms-full.txt` file that concatenates all important content into one document. This is simpler but less granular.
Markdown formatting best practices:
- Use clear heading hierarchy (H1 for page title, H2/H3 for sections)
- Include code blocks for technical content
- Keep paragraphs short (3-4 sentences max)
- Remove all navigation, footer, and sidebar content
- Include structured data like pricing tables as markdown tables
Step 3: Write Your llms.txt File
The file follows a specific markdown schema. Here is the structure:
Schema rules to follow:
- H1 header: Your brand or company name (only one H1)
- Blockquote: A concise summary of your company's purpose
- Key terms line: Important brand vocabulary that helps AI systems understand your domain
- H2 sections: Group content by category (Docs, Pricing, Policies, Support, etc.)
- Link format: `Title: Description` for each resource
- ## Optional section: Reserved for secondary content that can be skipped when context length is limited
Step 4: Deploy and Validate
Place the file at your domain root: `https://yourdomain.com/llms.txt`
Deployment checklist:
- The file must be publicly accessible (no authentication required)
- Serve it with a `text/plain` or `text/markdown` content type
- Do not block it in robots.txt
- Ensure it loads quickly (no server-side rendering delays)
- If you use a CDN, make sure the file is cached and served from the edge
Validation methods:
- Manual LLM test: Copy your llms.txt content into ChatGPT or Claude and ask: "Based on this file, what does this company do? What are its main products? What is its pricing?" If the AI gives accurate answers, your file is well-structured.
- llmstxt.org directory: Submit your file to the llmstxt.org directory for listing and basic validation.
- Link checking: Verify that every URL in the file returns a 200 status code and points to actual content (not redirects or error pages).
- Markdown linting: Run the file through a markdown linter to catch formatting issues that could confuse parsers.
Step 5: Set Up a Maintenance Schedule
An outdated llms.txt file is worse than no file at all. If your pricing changes, your API docs get restructured, or you launch new products, the file needs to reflect those changes.
Recommended maintenance cadence:
- Monthly: Quick review to check for broken links and outdated descriptions
- Quarterly: Full audit to add new content, remove deprecated pages, and update the key terms line
- On major launches: Update immediately when you ship new products, change pricing, or restructure documentation
llms.txt vs llms-full.txt: Which Should You Use?
The llms.txt specification includes two file types:
| Feature | llms.txt | llms-full.txt |
|---|---|---|
| **Purpose** | Curated index with links | Complete content in one file |
| **Size** | Small (1-5 KB typically) | Large (can be 100+ KB) |
| **Content** | Links to .md files with descriptions | Full text of all important pages concatenated |
| **Best for** | Sites with many pages, documentation-heavy products | Smaller sites, single-product companies |
| **Maintenance** | Update links when pages change | Regenerate entire file when any content changes |
| **AI parsing** | Requires following links to get full content | All content available in one request |
Our recommendation: Start with llms.txt (the index version). It is easier to maintain and gives you more control over what gets included. Only create llms-full.txt if you have a small site (under 20 important pages) or if you want to provide a single-file context dump for AI coding assistants.
llms.txt Templates for Different Site Types
SaaS Company Template
E-Commerce Store Template
Publisher or Content Site Template
Agency Template
How to Handle llms.txt on Different Platforms
The deployment method varies depending on your tech stack. Here are the most common approaches.
Static Sites (Next.js, Astro, Hugo, Gatsby)
Place the file in your `public/` or `static/` directory. It will be served automatically at the domain root.
- Next.js: Add `llms.txt` to the `public/` folder
- Astro: Add to `public/` folder
- Hugo: Add to `static/` folder
- Gatsby: Add to `static/` folder
WordPress
Two options:
- Plugin approach: Use a plugin like "LLMs.txt for WordPress" that auto-generates the file from your content
- Manual approach: Upload `llms.txt` to your WordPress root directory via FTP/SFTP (same level as `wp-config.php`)
If your WordPress site uses heavy page builders (Elementor, Divi), the markdown versions of your pages become especially important. Page builder HTML is notoriously difficult for AI systems to parse.
Shopify
Shopify does not allow files at the domain root by default. Workarounds:
- Custom proxy: Set up a Cloudflare Worker or similar proxy to serve the file at `/llms.txt`
- Subdomain: Host the file at `docs.yourstore.com/llms.txt` (less ideal but functional)
- Page-based: Create a Shopify page at `/pages/llms-txt` and link to it from your sitemap (non-standard but discoverable)
Custom Applications
If you control your server, add a route that serves the file with the correct content type:
How to Measure Whether llms.txt Is Working
This is the hardest part, and honesty matters here. As of August 2026, there is no definitive way to prove that llms.txt directly caused an improvement in AI search visibility. But there are signals you can track.
What to monitor:
- AI crawler access logs: Check your server logs for requests to `/llms.txt`. Look for user agents like `GPTBot`, `ClaudeBot`, `PerplexityBot`, `Googlebot`, and `Bingbot`. If crawlers are not requesting the file, it is not being read.
- AI search visibility metrics: Track your brand's visibility, citation rate, and share of voice across AI platforms over time. If you implement llms.txt alongside other GEO best practices, isolating its specific impact will be difficult, but directional trends still matter.
- Content accuracy in AI answers: Periodically ask ChatGPT, Gemini, Perplexity, and Claude questions about your product. Are the answers accurate? Do they reference current pricing? Do they describe your product correctly? If accuracy improves after implementing llms.txt, that is a positive signal (though not proof of causation).
- Citation URL patterns: If AI systems start citing the specific URLs you included in llms.txt more frequently, that suggests the file may be influencing retrieval. Tools like AI search visibility dashboards can track which of your URLs get cited most often.
Common llms.txt Mistakes (and How to Avoid Them)
1. Including every page on your site
The most common mistake. llms.txt is a curated index, not a sitemap. If you include 500 URLs, you have defeated the purpose. AI systems will either ignore the file or treat everything as equally important, which is the same as treating nothing as important.
Fix: Cap your file at 10-30 links. If a page is not something you would want an AI to cite when answering questions about your brand, leave it out.
2. Linking to cluttered HTML instead of clean markdown
If your llms.txt links point to pages full of navigation bars, cookie consent popups, JavaScript widgets, and sidebar ads, AI systems will struggle to extract the useful content.
Fix: Create dedicated .md versions of your key pages, or at minimum, ensure your HTML pages use semantic markup that AI systems can parse cleanly.
3. Setting it and forgetting it
Your pricing changed six months ago. Your API docs were restructured. You launched two new products. But your llms.txt still points to the old pages.
Fix: Add llms.txt to your deployment checklist. Whenever you update pricing, restructure docs, or launch products, update the file.
4. Using the wrong content type
Some servers serve .txt files with incorrect MIME types, which can cause parsing issues.
Fix: Ensure your server returns `Content-Type: text/plain` or `Content-Type: text/markdown` for the file.
5. Blocking the file in robots.txt
If your robots.txt disallows access to `/llms.txt`, crawlers that respect robots.txt will never see it.
Fix: Check your robots.txt to make sure there is no `Disallow: /llms.txt` rule. You can also add an explicit `Allow: /llms.txt` for clarity.
How llms.txt Fits Into Your Broader GEO Strategy
llms.txt is one small piece of a larger generative engine optimization strategy. It sits in the technical layer alongside structured data, semantic HTML, and crawl optimization.
Here is where it fits in the GEO stack:
- Content layer: High-quality, authoritative content that answers real user questions (this is where most of your effort should go)
- Technical layer: Semantic HTML, structured data (Schema.org), llms.txt, clean URL architecture, fast page loads
- Distribution layer: Third-party citations, reviews on G2/Capterra, Reddit and YouTube presence, backlinks from authoritative sources
- Measurement layer: Tracking visibility, citations, and share of voice across AI platforms
We can see from Superlines data, that citation patterns vary significantly across ChatGPT, Gemini, Perplexity, and Claude. Each platform has different retrieval mechanisms, which means no single technical optimization (including llms.txt) will work uniformly across all of them.
The companies seeing the best results in AI Search are the ones investing across all four layers, not just the technical one.
Implementation Checklist
Use this checklist to verify your llms.txt implementation is complete:
- [ ] Audited site for 10-30 highest-value pages
- [ ] Created clean markdown (.md) versions of key pages (or confirmed CMS outputs clean HTML)
- [ ] Written llms.txt with correct schema (H1, blockquote, H2 sections, ## Optional)
- [ ] Included key terms line with important brand vocabulary
- [ ] Each link has a brief description explaining what the page covers
- [ ] File deployed at `yourdomain.com/llms.txt`
- [ ] File returns 200 status code with correct content type
- [ ] File is not blocked by robots.txt
- [ ] Tested by pasting into ChatGPT/Claude and verifying accurate comprehension
- [ ] Submitted to llmstxt.org directory
- [ ] Set calendar reminder for monthly link check and quarterly full audit
- [ ] (Optional) Created llms-full.txt for single-file context dump
Start Building Your AI Content Foundation
llms.txt is a small but meaningful step toward making your site AI-readable. It will not transform your visibility overnight, and the data so far suggests its direct impact is minimal. But it costs almost nothing to implement, it forces a useful content audit, and it positions your site for whatever comes next as AI retrieval systems evolve.
The bigger opportunity is in the content itself. The brands winning in AI search are the ones producing authoritative, well-structured content that AI systems want to cite, not just the ones with the right technical files in place. If you want to understand where your brand currently stands across ChatGPT, Gemini, Perplexity, and other AI platforms, Superlines tracks your visibility, citations, and share of voice in real time, so you can see exactly which content is working and where the gaps are. Start a free trial to see your AI visibility baseline before and after implementing llms.txt.