Robots.txt Generator
Create a robots.txt file for your website to control search engine crawlers
Website Information
User Agents
Specify which search engine robots you want to control
Rules
Set up rules to allow or disallow access to specific parts of your site
Additional Options
Include Sitemap
Add Crawl Delay