Site audit

The site audit checks whether AI crawlers can reach and read your site, and whether your pages give engines what they need to quote you. It runs automatically every week, and you can run it any time from Site audit.

Site checks

Check	Why it matters
robots.txt allows the major AI crawlers	Blocked bots can't read or cite your pages.
Publishes /llms.txt	A short Markdown map of your key pages for AI agents. Cheap to add; not a ranking factor.
Has an XML sitemap	Helps search indexes (and the AI answers built on them) find and refresh pages.
Homepage has Organization structured data	States your name, logo and official profiles in a machine-readable way.

Page checks

Up to 30 pages are checked for:

Check	Why it matters
Page returns HTTP 200	Broken pages can't be cited.
Not marked noindex	Noindexed pages drop out of search indexes.
Main content is in the raw HTML	Most AI crawlers don't run JavaScript.
Has a title and meta description	Tells engines what the page answers.
Has exactly one H1	Clear structure makes passages easy to quote.
Has JSON-LD structured data	Product, Article, FAQ and similar schema state facts directly.
Has a canonical URL	Stops duplicates splitting signals.
Declares the page language	Signals the market the page serves.
Has Open Graph tags	Clean previews in chat apps and link unfurls.
Shows a published or updated date	Many prompts favour current sources.
Has at least ~300 words	Thin pages give engines nothing to quote.

Scores

• Page score — the weighted share of page checks a page passes. Access problems (HTTP status, noindex, JavaScript rendering) weigh most.
• Site score — the weighted share of site checks passed, with AI crawler access weighing most.
• Overall score — mostly the average page score, with site checks counting for 40%.

Fixing issues

Every failed check has How to fix with the reason, steps and a free tool to verify the fix. Top issues are also added to your action plan with priority based on how many pages fail.

For crawler access, the audit reads your robots.txt. Firewalls and CDN bot protection can still block AI crawlers even when robots.txt allows them — AI crawler tracking shows what actually reaches your server.