Robots.txt tester
Enter a URL to see if crawlers are allowed to fetch it, and view the site's robots.txt and sitemaps.
Run a full graded audit for the whole site.
Enter a URL to see if crawlers are allowed to fetch it, and view the site's robots.txt and sitemaps.