llms.txt Generator from Website Crawl

Inside this workflow

Overview

Convert website crawls into structured llms.txt format optimized for LLM training and RAG systems. Filters pages by relevance, summarizes content with LangChain, and generates a structured knowledge document. Nodes & apps: Form trigger, Filter node, Summarize node, LangChain LM Chat, Convert to File. Credentials needed: 1. OpenAI API key (for summarization) Setup steps: 1. Export your Screaming Frog crawl as CSV 2. Upload crawl to form trigger 3. Configure filter rules for relevant pages 4. Set LangChain summarization prompts 5. Configure output file format 6. Test with sample crawl 7. Deploy and process your website data Difficulty: Medium | Setup time: 15 minutes | Price: $29 | Nodes: 23 | Source: felipfr/awesome-n8n-workflows

What's included

Quickstart

1. 01

   Download the JSON

   With an active Library Pass, download the workflow file right from this page or your dashboard.

2. 02

   Import into n8n

   Choose Import from File and select the downloaded JSON. The full agent graph appears, ready to configure.

3. 03

   Plug in credentials

   Each integration node prompts for credentials on first run. The setup guide lists every credential the agent expects.

4. 04

   Activate and test

   Run once with sample input, confirm the expected output, then flip the activate toggle.

Sign in to leave a review.