AI readiness audit
Make sure AI assistants can actually read your site.
Before ChatGPT, Perplexity, Gemini or Google AI Overviews can cite you, their crawlers have to reach your pages, read the content and understand who you are. The AI readiness audit checks all three and tells you what to fix first.
Ask Brigham.aiAudit maid.co for AI readiness and tell me which AI crawlers we block, which pages they can’t read, and what to fix first.- AI crawlers blocked in robots.txt2 of 6
- Pages that need JavaScript to show text38
- Organization schemaPresent
- Articles without an author entity112
- llms.txt fileMissing
- 5
- AI assistants tracked
- Crawl
- Over crawl comparison
- $20
- Per month, every tool
- 7-day
- Free trial
Why readiness comes first
You can’t be recommended if you can’t be read.
- Part of
- Site audit
- Checks
- Access, rendering, schema, entities
- Compare
- Crawl over crawl
- Ask
- Brigham.ai
Most AI visibility advice starts with content, which assumes assistants can reach your pages. Often they can’t: a firewall blocks a crawler, an old robots.txt line turns away a bot, or the text only appears after JavaScript runs.
The search.co site audit includes a dedicated set of AI-readiness checks alongside its technical and on-page checks. It looks at your site the way an AI crawler does, then scores it and ranks issues by how many pages they affect.
Crawl access
Know exactly which AI bots you let in.
AI companies publish the user agents their crawlers use, and several run more than one: one for training data, one for search features, one that fetches a page when a user asks. The audit reads your robots.txt and tests each documented agent so you can decide on purpose, not by accident.
| User agent | Operated for | Your robots.txt | Fetch test |
|---|---|---|---|
| OAI-SearchBot | ChatGPT search results | Allowed | 200 OK |
| GPTBot | OpenAI model training | Blocked | Not tested |
| ChatGPT-User | Pages a user asks ChatGPT to open | Allowed | 200 OK |
| PerplexityBot | Perplexity search index | Allowed | 403 at firewall |
| Google-Extended | Gemini use of your content (token) | Allowed | n/a |
| ClaudeBot | Anthropic crawling | Blocked | Not tested |
01
Rules vs. reality
A bot can be allowed in robots.txt and still get blocked by a CDN or firewall rule.
02
Training vs. search
Blocking a training crawler is a business choice. Blocking a search crawler usually means you won’t be cited. The audit labels which is which.
Rendering
If the text needs JavaScript, some crawlers never see it.
Many AI crawlers fetch the raw HTML of a page and may not run JavaScript the way a browser does. If your product details, prices or answers are injected client-side, the crawler can come away with an empty shell.
The audit fetches every page twice, once as raw HTML and once fully rendered, and compares the text. Pages where most text only exists after rendering are flagged, grouped by template.
Raw vs. rendered word count
See how much of each page exists before any script runs.
Template grouping
One fix to a page template can clear hundreds of issues at once.
What the audit checks
Four groups of checks, one score.
Each check is a widely accepted practice. None guarantees a mention; together they remove reasons to skip you.
Access
4 checks- robots.txt rules for each documented AI user agent
- Firewall and CDN responses to those agents
- Pages blocked by noindex or nosnippet
- llms.txt present and well formed (optional)
Rendering
3 checks- Main text present in raw HTML
- Headings and lists in semantic markup
- Key facts not hidden in images or tabs
Structured data
3 checks- Organization or LocalBusiness schema
- Product, Service and FAQ markup where it fits
- Article schema with author and dates
Entities
3 checks- Consistent brand name, address and contact details
- Author pages with real bios
- sameAs links to official profiles
Entities
Help machines connect your name to what you do.
Assistants answer questions about companies, products and people. They do better when the facts about you are stated plainly and consistently: who you are, what you sell, where you work, who writes your content.
The audit compares how your brand is described across your pages, schema and linked profiles, and flags contradictions like two phone numbers or an old company name.
What you get
An audit you can hand to a developer.
Bot-by-bot access report
Every documented AI agent, your rule for it, and what happened when we fetched as that agent.
Ready-to-paste fixes
Suggested robots.txt lines, schema snippets and an llms.txt draft.
Grouped by template
Issues roll up by page type so one ticket covers every affected URL.
Crawl-over-crawl
Rerun after a release and see what closed and what’s new.
Change alerts
Email or Slack when a deploy blocks a crawler or drops your schema.
How it works
From crawl to ranked fixes.
- 01
Point it at your domain
Set the crawl limit and pick the AI agents you care about.
- 02
Let the crawl run
We fetch pages as each agent, render a sample and parse the structured data.
- 03
Review the score
See the overall score, each group’s score and the issues behind them.
- 04
Fix and recrawl
Ship the changes and rerun. The comparison shows exactly what improved.
Benchmark
See how your readiness compares to the sites assistants cite.
Readiness is relative. If the pages AI answers cite for your topics all have clean schema and server-rendered content, matching them is the baseline. Add competitors and the audit runs the same checks on their key pages so you can see the gap on each dimension.
Ask Brigham
Get the fix list in order of what it unblocks.
Perplexity’s crawler gets a 403 from your firewall on every page, even though robots.txt allows it. Your checklist page also loads its steps with JavaScript, so the raw HTML has a heading and almost no text. Until both are fixed, there is nothing for it to cite.
Recommended fixes
Sorted by impact
- 1
Allow PerplexityBot at the firewall
15 minYour bot-protection rule treats it as unknown traffic. Add an allow rule for its published user agent and IP ranges.
- 2
Server-render the checklist steps
Half dayMove the 24 steps into the HTML response. The same change applies to all 9 checklist pages.
- 3
Add HowTo-style structure and an author
1 hrUse numbered list markup, a visible author with a bio page and Article schema with dates.
- 4
Publish an llms.txt
10 minOptional and not used by every assistant, but a short map of your key pages is cheap to add. A draft is ready.
Should I block AI crawlers?
It depends on your goals. Some companies block training crawlers but allow the agents used for search and live answers, so they can still be cited. The audit shows what each agent is documented to do so you can make that decision deliberately.
Do I need an llms.txt file?
No. llms.txt is a proposed convention, and not every assistant reads it. It is cheap to add, so we flag it as a low-priority suggestion and can draft one from your pages, but it won’t fix access or rendering problems.
Does structured data get me into AI answers?
Nothing guarantees a mention. Structured data makes facts about your business, products and authors easier to parse and verify, which removes a common obstacle. We treat it as part of the baseline, not a shortcut.
Find out what AI crawlers see on your site.
Run your first AI readiness audit during the 7-day free trial. You’ll get a score, the blocked agents and a ranked fix list in one crawl.
Run an AI readiness audit on my site and list the three fixes that would unblock the most pages for ChatGPT, Perplexity and Gemini.