Skip to content
Enjoy 20% off for 12 months on all plans. Use promo code: HYPERGROWTHClaim it
Get your free plan

Our crawler

SearchBlueprintBot

The crawler behind SearchBlueprint's website audits: what it is, how to verify it, and how to allow or block it.

Contact us

At a glance

  • Names itself in its user agent
  • Follows robots.txt, including Crawl-delay
  • At most one request a second per site
  • Never opens sign-in, account or checkout pages

Identifying SearchBlueprintBot

Name

SearchBlueprintBot. robots.txt groups written for its earlier name, SEOAuditBot, are still obeyed.

Purpose

SEO, CRO, structured data and website auditing. It runs when someone starts an audit or a free check of a website, not on a schedule of its own, and only reads public pages.

User agent

Mozilla/5.0 (compatible; SearchBlueprintBot/1.0; +https://searchblueprint.io/bot). Every request carries this token. The Mobile CRO audit opens pages as a phone, with a phone browser's user agent ending in the same (compatible; SearchBlueprintBot/1.0; …) token.

Verification

Requests are not signed yet, so identify the crawler by its user agent. This page will link the public signing keys once signing is switched on.

IP addresses

None fixed: the crawler runs on cloud servers without dedicated addresses, so it is identified by its user agent, not by IP.

robots.txt

Every request follows robots.txt: the SearchBlueprintBot group if there is one, otherwise User-agent: *, including Disallow, Allow and Crawl-delay.

Rate

At most one request a second to one site, slower when robots.txt sets a longer Crawl-delay. The Technical SEO audit gives a site that answers 429 (or 503 with Retry-After) the pause it asks for before asking again.

Pages it never requests

Sign-in, registration, account, cart, checkout and admin pages (for example /wp-admin, /login, /account, /cart, /checkout), and Cloudflare's /cdn-cgi/ endpoints, whatever robots.txt allows.

How much it reads

A Technical SEO audit reads at most 300 pages of one site.

How to block it

In robots.txt: User-agent: SearchBlueprintBot then Disallow: /. SearchBlueprint never works around robots.txt.

Cloudflare

SearchBlueprintBot is not a Cloudflare Verified Bot yet (the application is in progress). Owners can allow it themselves, or connect Cloudflare inside SearchBlueprint so it creates a narrow, temporary exception and removes it after the audit.