• Installing the Magento 2 LLMs.txt Generator extension
• Set up cron
• Uninstalling the Magento 2 LLMs.txt Generator extension
• Extension Compatibility
• Introducing to Magento 2 LLMs.txt Generator
  • Extension Overview
    • Key Features
• Using Magento 2 LLMs.txt Generator
  • Extension Configuration
  • Managing Content for LLMs
    • Main Product Categories
    • Commercial & Service Information (CMS Pages)
  • File Generation & Multi-Store Architecture
    • Multi-Store Setup
    • File Structure & Smart Chunking
    • Generation via Command-Line Interface (CLI)
    • File Locations & Accessibility

Installing the Magento 2 LLMs.txt Generator extension

Command Line Installation

1. Backup your web directory and store database

2. Download the LLMs.txt Generator installation package

3. Upload contents of the LLMs.txt Generator installation package to your store root directory

4. In SSH console of your server navigate to your store root folder:

cd path_to_the_store_root_folder

Run the following command: 

php bin/magento module:enable Aheadworks_LLMsTxtGenerator

then:

php bin/magento setup:upgrade

after:

php bin/magento setup:static-content:deploy -f

5. Flush store cache; log out from the backend and log in again

Composer Installation

If you are installing an extension from Aheadworks for the first time, you need to add our composer repository to your Magento store:

   1. Login to your ssh console and navigate to your store folder:

cd path_to_the_store_root_folder

Run the following commands:

After the command was executed successfully, you can use the composer to install the extension.

To install the extension:

   2. Login to your ssh console and navigate to your store folder:

cd path_to_the_store_root_folder

   3. Run the following command to install the latest version of the extension:

composer require aheadworks/module-llms-txt

if you need to install a specific version, run this command:

composer require aheadworks/module-llms-txt<version>

When prompted, enter Public Key (Username) and Private Key (Password):

Both Public Key (Username) and Private Key (Password) can be found in My Projects and Licenses in your personal account on our site:

 To upgrade the extension: 

To update or upgrade an extension:

Download the updated extension file. Take note of the module-name and version. Export the contents to your Magento root.

If a composer package exists for the extension, run one of the following.

Update per module name:

composer update aheadworks/module-llms-txt

Updater per version:

composer require aheadworks/module-llms-txt:<version>

Run the following commands to upgrade, deploy, and clean the cache.

php bin/magento setup:upgrade --keep-generated
php bin/magento setup:static-content:deploy
php bin/magento cache:clean

Set up cron

If you have already configured cron jobs for your Magento installation then you can skip this step. Read the following instructions on setting cron job for your Magento store: Magento user guide.-

Generally, it would be enough to run in SSH console of your server: 

crontab -e

And insert the following line:  

*/3 * * * * php -c <ini-file-path> <your Magento install dir>/bin/magento cron:run
*/3 * * * * php -c <ini-file-path> <your Magento install dir>/update/cron.php
*/3 * * * * php -c <ini-file-path> <your Magento install dir>/bin/magento setup:cron:run

Remember to confirm the Save request when exiting the installation.

Uninstalling the Magento 2 LLMs.txt Generator extension

Since Magento introduced declarative Database you need to be very careful when uninstalling or disabling the module.

Manual Removal

1. Disable the module by executing the following commands:

php bin/magento module:disable Aheadworks_LLMsTxtGenerator
php bin/magento setup:upgrade

2. Remove the extension files from the following folder:

app/code/Aheadworks/LLMsTxtGenerator

Automatic Removal (via Composer)

1. Disable the module by executing the following commands:

php bin/magento module:uninstall Aheadworks_LLMsTxtGenerator

Extension Compatibility 

Introducing to Magento 2 LLMs.txt Generator

Extension Overview

The LLMs.txt Generator for Magento 2 extension bridges the gap between your e-commerce storefront and the emerging ecosystem of AI crawlers, LLM agents, and smart search engines like ChatGPT, Perplexity, Gemini, and Claude. Inspired by robots.txt and sitemap.xml, this module automatically generates and maintains structured plain-text files (llms.txt and llms-full.txt) in your site root. It serves as a secure, high-performance gateway for generative AI models to discover, index, and parse your product catalog without straining server resources or exposing sensitive store paths.

Optimizing your store for AI search discovery (AEO & GEO) has never been easier. The module intelligently maps Magento categories, products, and CMS pages while giving you complete administrative control over indexed data, automated HTML markup stripping, and custom plain-text AI descriptions. Built to handle enterprise-level catalogs, native multi-store architecture delivers localized files per Store View, while smart automated file chunking keeps your site fast, stable, and completely safe from server memory timeouts.

Key Features

• Automatically generate llms.txt and llms-full.txt files for fast AI web crawler indexing.

• Configure custom business models, tax policies, currencies, and shipping rules for AI search engines.

• Fine-tune AI visibility and automatic HTML-free plain-text descriptions for categories and CMS pages.

• Safeguard store resources by blocking AI agents from indexing checkout paths, customer accounts, and API routes.

• Generate localized, multi-store files with dedicated languages and dynamic URLs per Store View.

• Prevent server timeouts by automatically splitting large product catalogs into lightweight file chunks (~125–128 KB).

Using Magento 2 LLMs.txt Generator

Extension Configuration

To configure the general settings of the LLMs.txt Generator extension, navigate to Stores → Configuration → Aheadworks Extensions → LLMs.txt.

General

This section allows you to customize the baseline store information that will be rendered at the top of the generated file.

• Store Name – Specifies the store name used as the main H1 heading in the llms.txt file.

• Store Information – Allows you to add custom markdown content placed directly below the store name (e.g., Legal Entity, Physical Address, and Support Contact). HTML is not allowed in this field; use standard Markdown format only.

AI Agent Directives & Core Rules

These settings provide explicit context for AI agents and crawlers regarding your business model, pricing, and shipping terms.

• Target Audience – Defines the primary audience for your store (B2B, B2C, or B2B and B2C).

• Default Currency – Specifies the currency displayed to AI agents in the llms.txt file.

• Tax Policy – Clarifies your tax application rules (e.g., Prices exclude tax) so crawlers don't misinterpret pricing data.

• Shipping Regions – Allows you to outline your shipping availability and geographical coverage using custom text/Markdown.

• Order Restrictions – Mentions any purchasing limits or constraints for buyers. You can write custom constraints like age-restricted items, complex wholesale rules, or pack-size limits when Use Default is unchecked.

Crawling Restrictions

This section allows you to manage the boundaries for AI agents, specifying which parts of your store should remain off-limits to prevent resource wasting or parsing of sensitive data.

• Crawling Restrictions – Displays the technical directives and URL patterns that LLMs must not crawl or parse (e.g., shopping cart, checkout, user account pages). This protects your dynamic routes and keeps AI agents away from sensitive endpoints.

By default, these rules are built automatically from your core Magento routes and store-view URL rewrites for the selected scope. Unchecking Use Default allows you to manually adjust these restrictions or add custom paths.

Managing Content for LLMs

The extension populates the data files by pulling records directly from two main Magento components: catalog categories and active CMS pages.

Main Product Categories

New products are added to the index files automatically. The extension only includes products that are set to Enabled with visibility configured to Catalog, Search. If a product is assigned to multiple categories across the store, the extension filters the entries and records the item exactly once using its canonical URL.

To configure category-specific AI settings, navigate to Catalog → Categories, select the target category, and open the AI optimization tab.

• Include in LLMs.txt – A toggle switch allowing you to manually include or exclude a specific category from the llms.txt file.
  When left on default, the extension automatically determines inclusion based on the category status, menu visibility settings, and maximal depth rules.

• AI Description – Allows you to input a custom, plain-text summary for AI agents.
  When the Use Default checkbox is active, the extension inherits the text from the standard category Description. However, if the main description contains HTML tags, the module automatically falls back to pulling data from the category's Meta Description to ensure clean text delivery.

Commercial & Service Information (CMS Pages)

To configure category-specific AI settings, navigate to Content → Pages, locate the target CMS page in the grid, click Select under the Action column, and choose Edit.

Expand the AI optimization section on the page edit screen to configure page-specific LLM parameters.

• Include in LLMs.txt – A toggle switch allowing you to explicitly include or exclude the selected CMS page from the generated text files.

• AI Description – A plain-text field where you can provide a dedicated summary of the page specifically for LLMs and AI crawlers.
  When the Use Default checkbox is active, the extension automatically strips out HTML markup, headers, and UI elements from the main page content, extracting clean text to ensure AI agents read accurate business descriptions.

File Generation & Multi-Store Architecture

Multi-Store Setup

The extension fully supports multi-store configurations, generating accurate, localized text files tailored to each specific Store View.

• Localized Content: Separate files are generated to reflect unique product availability, localized URLs, and store-specific descriptions.

• Inheritance: If a secondary store view does not have custom AI settings or categories configured, the generated file automatically inherits values from the default scope.

File Structure & Smart Chunking

To optimize crawling speed and prevent payload issues with large product catalogs, the module divides data across specialized files:

Generation via Command-Line Interface (CLI)

Files are generated and refreshed manually using terminal commands. The generator indexes real-time catalog states, category visibility, and AI settings at the moment of execution.

Navigate to your Magento root directory and run the appropriate command:

• Generate files for all Store Views simultaneously:
   
  bin/magento aw_llms_txt:generate

Recommended for initial module setups or after bulk catalog updates.

• Target a specific Store View (e.g., Store View ID):
  
  bin/magento aw_llms_txt:generate --store=1

Ideal for updating an isolated localized store without re-indexing the entire multi-store infrastructure.

File Locations & Accessibility

Upon executing the CLI command, the extension places generated files directly into your web-accessible directory according to the Store View scope:

• Default / Root Scope

Files are saved in the pub/ directory (or web root) and accessible via the root domain:

[https://yourstore.com/llms.txt]

[https://yourstore.com/llms-full.txt]

• Localized Store View Scopes

For secondary store views, files are organized into subdirectories matching the Store View code:

[https://yourstore.com/uk/llms.txt]

[https://yourstore.com/de/llms.txt]