create-llmstxt-py

The Firecrawl LLMs.txt Generator is a Python script designed to create `llms.txt` and `llms-full.txt` files for any specified website using Firecrawl and OpenAI APIs. Its primary use case is to enhance the accessibility of website content for Large Language Models by automatically mapping URLs, extracting markdown content, and generating summarized titles and descriptions through AI. Notable features include parallel processing for efficiency, customizable URL limits, and flexible output options to suit user needs.