🕷️ Free · no signup · no app install · read-only
Can ChatGPT and Perplexity actually read your store?
Almost nobody edits robots.txt on purpose — but a theme edit, an SEO app or a leftover staging rule can quietly ban the crawlers that feed AI shopping answers. We read your robots.txt and your served HTML and tell you, agent by agent, who gets in.
Based on publicly available data only. Informational, not legal advice.
What this checks
- 14 named AI and search crawlers, each judged against your home page and your product pages
- A site-wide Disallow: / — the one line that hides the whole store from everyone
- Whether /products/ is blocked for the default User-agent: * group
- Whether /llms.txt exists, and whether it is a real text file rather than a 404 page
- Whether robots.txt declares a sitemap, and whether that sitemap actually loads
- Whether product links and prices are in the HTML the server returns, or only appear after JavaScript runs
- A noindex in the page meta tags or the X-Robots-Tag header
FAQ
If a crawler is allowed, does that mean ChatGPT will recommend my products?
No. Being crawlable is the entry ticket, not the outcome. This tool tells you whether you have disqualified yourself on a technicality — it cannot tell you how any model ranks or cites you.
Why do some lines say 'could not tell'?
We fetch pages the way a plain crawler does and never run JavaScript. If robots.txt does not load, or the storefront blocks our request, we mark the affected checks unknown with the reason attached instead of guessing a pass. robots.txt is also honour-system: it says what compliant crawlers are asked to do, not what every bot actually does.
Is blocking AI crawlers always wrong?
No, and we do not score it that way. Blocking training-only opt-out tokens such as Google-Extended is a legitimate choice with a known cost. What is almost never intended is blocking the agents that fetch a page live because a shopper just asked about you — that one we flag hard.
Want robots.txt unblocked, a real llms.txt shipped, and a monthly check that it stays that way?
We are a Shopify studio. The audit is free; the fix is yours to make — or ours to ship.