Robots.txt generator

Build a valid robots.txt with allow/disallow rules, crawl-delay and sitemap lines. Then test it with the robots.txt tester.

User-agent

Allow (one path per line)

Disallow (one path per line)

Crawl-delay (optional)

Sitemap URL(s)

robots.txt

User-agent: *
Allow: /

Sitemap: https://example.com/sitemap.xml

Auditing the whole site? Run a full audit