See Your Page as an AI Crawler Sees It

The tool requests your URL with the GPTBot user-agent and reads only the HTML the server sends back. No scripts run, no client-side data loads, no cookie banners are dismissed. It then extracts the main content (from <main>, <article> or <body>, in that order) and turns it into Markdown, which is close to how retrieval systems simplify a page before a model reads it.

It also sends the same request as ClaudeBot, PerplexityBot and a normal Chrome browser. If the browser gets a 200 and a bot gets a 403, a firewall or CDN rule is treating AI crawlers differently — something robots.txt can't show you.

• Content without JavaScript fails when the server HTML holds fewer than 50 words. That almost always means a client-rendered app. Fix it with server-side rendering or static generation.
• Access by bot identity compares HTTP codes. A mismatch between the browser and a bot points to WAF or bot-management settings.
• Readable content counts words in the main region. Under roughly 300 words there is little for an engine to quote.
• Text density is readable text as a share of markup (scripts and styles excluded). Modern component-based sites commonly sit between 1% and 9%, so only very low values are flagged.
• The token estimate uses about four characters per token, a common rule of thumb for English with GPT-style tokenizers. Use the token counter for other text.

Google renders JavaScript for its search index, so a client-rendered site can rank in Google while being nearly invisible to other AI crawlers. When ChatGPT, Claude or Perplexity look for sources, they can only quote what was in the HTML they fetched. If your pricing, product details or answers load after hydration, those facts are missing from the version AI engines see.

The Markdown panel is the quickest sanity check: if the key sentence you want AI to repeat about your brand isn't in it, no engine will repeat it from this page. Once the page reads well here, run the AEO checker for schema and meta signals.