Getting Your Website Found in ChatGPT: Implement "LLMs.txt"

The way people find information online is changing at lightning speed. While traditional search engines like Google have dominated the field for years, AI-driven systems such as ChatGPT, Claude, and Perplexity are increasingly taking the lead. These Large Language Models (LLMs) generate answers based on web content but struggle to navigate complex websites filled with advertisements, navigation menus, and other noise.

To address this challenge, Jeremy Howard, co-founder of Answer.AI, introduced the concept of the llms.txt file in September 2024. This simple text file, placed in the root of your website, provides AI systems with a structured overview of your most important content. In this article, you will discover what llms.txt is, why it is relevant for entrepreneurs and marketers, and how to implement it to increase your visibility in AI-driven search results.

What is llms.txt?

The llms.txt file is a Markdown document that you place in the root of your website, for example at yourdomain.com/llms.txt. In this file, you provide an overview of your most valuable content, such as product pages, manuals, policy documents, and blog articles. By offering this information in a structured manner, AI systems can more efficiently and accurately understand your website and cite relevant content.

It is important to know that llms.txt is not a replacement for existing files like robots.txt or sitemap.xml, but rather a supplement to them. While robots.txt provides instructions on which parts of your site should not be crawled and sitemap.xml offers an overview of all pages, llms.txt acts as a sort of table of contents specifically for AI systems.

Why is llms.txt relevant for entrepreneurs and marketers?

1. Increased visibility in AI answers: AI systems increasingly use web content to generate answers. By structuring your core content in an llms.txt file, you increase the chance that your information will be included in AI responses.
2. Better control over your brand story: With llms.txt, you determine which content AI systems consider representative of your brand, giving you more control over how your company is presented in AI-generated content.
3. More efficient content processing: AI systems have limited capacity to process information. By providing a clear overview of your most important content, you help AI systems find relevant information faster and more accurately.

How do you implement llms.txt?

1. Identify your core content: Determine which pages are essential for your brand and what information you want AI systems to use.
2. Apply structure: Use Markdown to structure your content. Start with a title and a brief summary of your website. Then use headings (e.g., ## Products, ## Manuals) to categorize your content and list a brief description and the corresponding URL for each item.
3. Place the file on your website: Upload the llms.txt file to the root of your website so it is accessible via https://yourdomain.com/llms.txt.
4. Maintenance and updates: Ensure your llms.txt file stays up-to-date by regularly checking if links still work and if the content remains relevant.

What are the limitations of llms.txt?

While llms.txt offers many benefits, there are also some limitations:

• Not yet an official standard: llms.txt is currently a proposal and not yet an officially recognized web standard. However, it is suggested that OpenAI, Anthropic, and Microsoft all utilize it when searching websites.
• Maintenance required: The file must be updated regularly to remain relevant, which requires additional maintenance.
• No guarantee of use by AI systems: Language models are "non-deterministic," meaning there is no guarantee that you or someone else will receive the same answer to the same question. Therefore, you might be visible to one user but not necessarily see it yourself every time.

Who is already using llms.txt?

Various companies and organizations have already implemented llms.txt, including:

• Anthropic: Known for the AI assistant Claude, Anthropic has integrated llms.txt to make their content more accessible to AI systems.
• Mintlify: A platform for developer documentation that uses llms.txt to offer their content in a structured way.
• Daxko: A technology company that uses llms.txt as part of their SEO strategy for AI search engines.

Here you can find a list of companies and their llms.txt files, where you can click through for inspiration regarding structure.

Conclusion

In an era where AI-driven systems play an increasing role in how information is found and presented, llms.txt offers a simple yet effective way to make your content more accessible to these systems. By providing your most important information in a structured way, you increase the likelihood that your content will be included in AI answers, leading to greater visibility and authority for your brand.

For entrepreneurs and marketers, it is therefore wise to invest now in setting up and maintaining an llms.txt file. It is a relatively simple step that can help your brand remain visible and relevant in the rapidly evolving landscape of AI-driven information delivery.

// About the author

Remy Gieling

Mede-oprichter, AI-expert & bestseller-auteur

Tech-expert (1988) gespecialiseerd in kunstmatige intelligentie en mede-oprichter van ai.nl, The Automation Group, Proxies en eBrain.ai. Oud-hoofdredacteur van diverse zakenmerken en daardoor een geoefend verteller op het podium en in de media. Verzorgt jaarlijks 150+ AI-keynotes in binnen- en buitenland en is gastdocent aan Nyenrode. Co-auteur van zeven boeken, waaronder 'Handboek AI Strategie' en 'AI Agents', en bekend als presentator op radio en RTL Z. Reist langs de labs van OpenAI, Nvidia en Tencent en vertaalt de nieuwste doorbraken naar inzichten die leiders direct kunnen toepassen.

LinkedIn