Agentic
llms.txt llms-full.txt posts.json rss.xml sitemap.xml robots.txt

Robots.txt Generator

$ robots-txt-generator --free

Visual builder with allow and disallow rules per user-agent.

Rule #1
Preview 
User-agent: *
Disallow: