Robots.txt on Squarespace: How to Check and Fix It
By Toy Wen Da, Co-Founder · Updated · 7 min read · scored 99/100
A robots.txt squarespace file is the text file Squarespace generates automatically at yoursite.com/robots.txt. It tells search engines and AI crawlers which parts of your site they may request. You can view it in any browser but you can't edit the raw file. Your real levers are a few settings under Settings and each page's SEO panel.
Check it first with a free tool
Before you dig through settings, run your domain through the free Robots.txt Tester. Paste in your URL and it shows which directives are live, whether a specific page or bot is blocked, and whether the syntax Squarespace generated is valid. Since you can't edit the file by hand, a quick external check is often faster than hunting through menus to confirm nothing is silently blocking Googlebot.
What a Squarespace robots.txt file actually contains
Every Squarespace site ships with a default robots.txt that disallows a handful of system paths, such as checkout pages, search result pages and admin routes, while leaving your real content open to crawling. You didn't write any of it, and you can't remove or add lines directly.
On a self-hosted site you'd have full file access. Here the file is generated behind the scenes and updates automatically as you add pages, collections or e-commerce sections.
Why Squarespace locks this down
Google's documentation notes that if you use a content management system, you likely won't edit robots.txt directly. The CMS exposes a settings page or toggle instead, and Google asks you to manage crawling through that interface (Google Search Central). Squarespace follows that pattern: settings and page-level toggles stand in for raw file access.
How to view your Squarespace robots.txt file
Viewing it takes a few steps:
- Open a browser and go to
yoursite.com/robots.txt, using your own domain in place of "yoursite.com". - Confirm there's no
Disallow: /line underUser-agent: *, because that single line would block every crawler from your entire site. - Scan for any paths you didn't expect to see disallowed.
If the file looks fine but a specific page still isn't showing up in Google, robots.txt usually isn't the culprit. Check that page's own SEO settings instead.
Can you edit robots.txt on Squarespace?
Not directly. There's no Code Injection field or settings panel where you can paste a custom robots.txt and have it override the generated one. What you can do is change the settings that influence what the file, and the rest of your crawl behavior, allows.
The settings that actually move the needle
| Setting | Where to Find It | What It Controls |
|---|---|---|
| Block known AI crawlers | Settings → Crawlers | Adds disallow rules for bots like GPTBot, ClaudeBot and similar AI crawlers |
| Hide page from search results | Page Settings → SEO (per page) | Adds a noindex instruction for that one page, separate from robots.txt |
| Password protection | Page Settings → General | Blocks both crawlers and visitors from a page entirely |
| URL Mappings | Settings → Advanced → URL Mappings | Redirects one path to another; doesn't rewrite robots.txt itself |
If you want AI tools like ChatGPT or Perplexity to crawl your content, leave the "Block known artificial intelligence crawlers" checkbox off. If you'd rather keep those bots out while still allowing Google and Bing, switch it on. It targets AI crawlers specifically and leaves standard search engines alone.
If you need full file control
Some sites need custom crawl-delay rules, wildcard patterns or bot-specific sections that Squarespace's toggles can't produce. The practical path then is to build the exact file you want with the free Robots.txt Generator and move to a platform that lets you upload it at the root. Webflow and Framer both support this, and Squarespace does not.
Crawl-delay rules rarely do what people expect even on platforms that support them. Googlebot ignores the directive outright, as covered in Robots.txt Crawl-Delay: Why Googlebot Ignores It, so switching platforms for that one directive usually isn't worth it.
When Squarespace's built-in tools are enough, and when they're not
| Situation | Squarespace's Tools Enough? | What To Do |
|---|---|---|
| One page not indexing | Yes | Check that page's SEO settings for an accidental noindex toggle |
| Want to block AI crawlers site-wide | Yes | Flip the AI crawler checkbox in Settings → Crawlers |
| Need a custom crawl-delay or wildcard rule | No | Squarespace won't expose this; a platform with file access is required |
| Recurring crawl errors across many pages | Partially | Settings catch common cases; deeper issues need Search Console review |
| One-off check before a launch | Yes | A quick look at /robots.txt plus the tester covers it |
A single blocked URL is almost always a settings fix. A recurring, site-wide problem is different: pages dropping out of the index in waves, or bots hitting pages you never configured. Either one points to something structural that Squarespace's panel can't touch.
Common mistakes with Squarespace robots.txt
- Confusing noindex with disallow. A disallowed page can still get indexed if other sites link to it, while a noindex tag is what keeps a page out of results.
- Assuming the AI crawler toggle affects Google. It targets AI bots only, and Googlebot isn't part of that switch.
- Password-protecting a page to hide it from search, then wondering why traffic drops. Password protection blocks crawlers entirely, not just one feature.
- Expecting a crawl-delay rule to slow Googlebot down. Google ignores that directive regardless of platform.
- Checking robots.txt on a staging URL instead of the live domain, which can show different rules entirely.
Check your fix with the free Robots.txt Tester
After you adjust the AI crawler setting or a page's SEO toggle, publish your changes and run the domain through the Robots.txt Tester again. It confirms the live file reflects your change and flags any path that's still unexpectedly blocked. That helps because Squarespace doesn't show a before-and-after diff when you flip a setting.
FAQs
Does Squarespace let me upload my own robots.txt file?
No. Squarespace generates the file automatically based on your site structure and a small set of settings. There's no field to paste or upload a replacement.
Will blocking AI crawlers in Squarespace also block Google?
No. The "Block known artificial intelligence crawlers" setting targets bots like GPTBot and ClaudeBot specifically. Googlebot and other standard search crawlers aren't affected by that toggle.
Why is my page blocked in robots.txt but still showing up in Google?
Google can index a URL from external links even without crawling its content, which is why disallowed pages sometimes appear in results without a description. Use a noindex tag if you need a page fully out of the index.
Can I add a sitemap reference to my Squarespace robots.txt?
Squarespace already includes your sitemap reference in the generated file, so you don't need to add it yourself.
Is llms.txt the same thing as robots.txt on Squarespace?
No. Robots.txt controls crawler access and has been a web standard for decades, formalized by the IETF as RFC 9309. Squarespace's newer llms.txt feature is an unofficial, experimental file format that major AI crawlers don't currently rely on for discovery or citation. It's a separate and much smaller concern than getting robots.txt right.
Get a second set of eyes on your crawl settings
Robots.txt on Squarespace is a small file that can quietly cost you traffic if one toggle gets flipped the wrong way. If you want a fuller picture of how your site is crawled and indexed than a single file can give, talk to the team and ask for a sample report showing where your site stands.
Resources / References
Sources
About the author
SearchBlueprint researched, wrote and scored this article with the content pipeline our customers run on their own sites. The score in the byline is what our content engine gave this page before it published. Nothing here is reviewed by hand, which is the point: it is the product's output, published where anyone can check it.