Check any page against ChatGPT, Perplexity, Claude and Google AI.
Example page
Switch views to see what a crawler receives.
Burr espresso grinder
$249.00
4.8 out of 5 stars from 312 reviews
Flat steel burrs and 40 grind settings, from fine espresso to coarse cold brew. Grinds a double shot in about ten seconds.
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Shop</title>
<link rel="stylesheet" href="/app.css">
</head>
<body>
<div id="root"></div>
<script src="/app.js"></script>
</body>
</html>Readable text: 0 words
Content hiddenA person sees the product. A crawler that does not run JavaScript sees an empty page.
GPTBot · OAI-SearchBot · ClaudeBot · PerplexityBot · Google-Extended · Applebot-Extended
Four common causes
The cause is often technical, and technical causes can be found and fixed.
One old rule, or one added by a plugin, can block GPTBot, OAI-SearchBot, ClaudeBot or PerplexityBot from the whole site. It is easy to miss because people can still visit every page.
User-agent: GPTBot
Disallow: /
User-agent: PerplexityBot
Disallow: /Bot protection on Cloudflare, Akamai or a WAF often answers automated visitors with a challenge instead of your content. People get through, while crawlers receive a page with nothing to read.
HTTP/2 403
cf-mitigated: challenge
content-type: text/htmlMany AI crawlers read the raw HTML and do not run scripts. If your text is rendered in the browser, what they receive is an empty shell.
<body>
<div id="root"></div>
<script src="/app.js"></script>
</body>Missing titles and headings, no main landmark and conflicting canonicals make a page harder to interpret. Absent or broken structured data hides who the page belongs to and what it offers.
<h1>: missing
<main>: missing
JSON-LD: parse error at line 4Four steps
From a free page check to a site you know stays readable.
Paste one page address. You get a result right away: whether AI crawlers can reach and read it, most serious findings first.
Up to 500 pages are crawled twice, once as a browser sees them and once as a non-rendering crawler does, then compared page by page.
A person reads every report before delivery. It lists findings with evidence, affected pages and fixes in priority order.
Your team applies the fixes, or we can. Request the included recheck from the report page within 14 days to see which findings are resolved, new or still open. A monthly retainer keeps checking after that.
Four scored categories
Deterministic checks in four weighted categories. The score is computed by rules, not by AI, so the same site gives the same result every time.
Weight 40%
Weight 30%
Weight 20%
Weight 10%
Claude, Anthropic’s model, is used for one job: turning verified findings into plain-language explanations and fix steps. It cannot add a finding, remove one or change its severity, and it never sees your raw HTML. A training opt-out never lowers your search-readiness score.
Example report
Each finding comes with its evidence, how many pages it affects, and the change that fixes it. Template issues are shown once with example URLs, not hundreds of times.
shop.example.com
Full-site audit, 412 of 430 discovered pages checked
Readiness score
58/ 100
Coverage 92%
Share of applicable checks that could be evaluated.
The score measures published checks, not the probability of being cited.
robots.txt blocks AI search crawlers
1 file: /robots.txt, homepage blocked
bots: OAI-SearchBot, PerplexityBot
rules[1].matchedRule: Disallow: /Raw HTML holds less than half of the rendered text
138 pages
ratio: 0.07
rawChars: 286
renderedChars: 4120Structured data is missing required fields
1 page
missing: LocalBusiness.addressFive offers
Audits are paid before the crawl runs and check up to 500 pages. On a larger site the report shows how many of the pages found were checked, so ask us first. Fixes are always quoted from the audit, never bundled blind.
$0one page
One public URL, instant result, no account.
Recommended first step
from $300per site
Up to 500 pages, deterministic checks, human review, a report page and PDF with evidence and prioritized fixes, and one recheck you request from the report within 14 days. Typically $300 to $500.
from $800per site
Robots, CDN and WAF rules, content moved into server HTML, schema, llms.txt and template fixes. Quoted from your audit, typically $800 to $2,000.
from $150per site, per month
Monthly recheck, a diff against the last report, and a verified AI-crawler visit report from your server or Cloudflare logs. Typically $150 to $300.
from $150per audit
For SEO consultants and agencies. The same audit, delivered to you to resell. Typically $150 to $250.
Prices in US dollars. Payment by invoice through Wise, Payoneer or a Stripe payment link.
Eight questions
No one can promise that, and we do not. An audit removes technical barriers that stop AI tools from reaching and reading your pages. Whether an assistant then cites you depends on many things outside your site.
One public page: its status and redirects, the robots.txt rules for each AI crawler, robots meta and headers, whether the main text is in the raw HTML, and basic structure. The full audit runs the same checks across up to 500 pages and compares raw and rendered versions of each.
No. A fixed set of rules produces every finding and the score, so the same site gives the same result. Claude only writes the plain-language explanations and fix steps from findings the rules have already verified.
No. We read what any public visitor or crawler can read. For the monthly retainer's crawler visit report, you share server or Cloudflare logs if you want that part.
The crawl starts once the invoice is paid and is capped at 500 pages; on a larger site the report says how many of the pages found were checked. A person then reviews the report before delivery. We confirm the delivery date when we send the invoice.
Then the report says so. When we cannot fetch enough of the site, we show Incomplete or Access blocked instead of a score, together with what blocked us. If your robots.txt disallows CrawlitoBot by name, Crawlito respects it and stops before fetching any page; the About CrawlitoBot page explains how.
No. We label every crawler by purpose: search, training or user-triggered. Opting out of training never lowers your search-readiness score.
Yes. The agency wholesale audit carries your name, logo and colours instead of Crawlito's, on the report page and the PDF, and is delivered to you to send on. Request a fix on the report emails you, or you can have it hidden, so your clients never contact us.
Start with one page. The check is free and needs no account.
Check a page