Access checks

The Access tab in AI Readiness answers one question: can AI engines get to your content? It holds four checks. Each opens into a page with an Overview tab (this week's state) and a Timeline tab (the weekly runs side by side, so you can see when something changed).

---

Reading a check

• Status pills. Pass, Allowed or Healthy when all is well; Warning for something worth a look; Blocked, Fail or Critical when AI engines are being kept out; Info for a note that needs no action; Unknown when the check could not get an answer.
• Findings. Robots.txt and Sitemap list their findings with a severity, searchable and filterable. Click one for Why this matters, the Evidence behind it and How to fix it. Crawlability and Edge / WAF show a row per bot instead, each with its own detail.
• Snippets. Where a fix is a piece of configuration, Copy puts it on your clipboard to send to whoever runs the site.
• What already passes. Each check lists the checks passing, so you know what has been verified.
• Next audit. The header shows when the check runs next.

---

Crawlability

Can the named AI bots fetch your homepage? The card shows how many of the checked bots can.

Each bot is tested against three signals, in order:

1. robots.txt: does your robots.txt allow it?
2. Fetch with bot User-Agent: does your homepage answer normally (a 2xx response) when requested as that bot?
3. Meta directives / X-Robots-Tag: does an on-page tag or a response header tell it to stay away?

Bots are labelled by what they do, because blocking some costs more than others:

• User-prompted: fetches a page because a user asked the assistant about it. Blocking these hurts most.
• Answer engine: powers the engine's search and answers.
• Training: collects content for model training. Blocking these is often a deliberate opt-out.

Search the table by bot name, or filter to All, Issues only or Allowed. The Timeline shows a week-by-week grid per bot, with what flipped and when.

If your robots.txt cannot be reached during an audit, every bot's robots.txt signal reads Unknown until the next successful fetch.

---

Robots.txt analysis

Is your robots.txt valid, and is it doing what you intend? GEOflux validates the file for syntax, sitemap declarations and conflicting rules. The card shows the number of issues.

• Findings point to the exact line or lines. View file shows your robots.txt as GEOflux fetched it.
• The Timeline tracks the file week over week. A week marked File changed means someone edited it, and the line-by-line difference shows whether the edit introduced or fixed a problem.

---

Sitemap reachability

Are the pages AI cites in your sitemap, and is the sitemap healthy? The card shows how many tracked pages the sitemap covers, and how many issues it found.

• Coverage compares your tracked pages with the sitemap and lists the missing ones. Tracked pages are pages on your own domain that AI answers cited for your prompts over the last 90 days (up to 100), marked Prompt-linked.
• Structural findings check the sitemap for freshness, structure and consistency, such as duplicate entries or a broken sub-sitemap.
• The overview shows how the sitemap was found: Declared in robots.txt, or Found via fallback when robots.txt does not declare one.

The Timeline shows how coverage and findings changed week to week.

---

Edge / WAF

Is something in front of your site turning AI bots away? CDNs and firewalls can block bots before a request ever reaches your server, and nothing in your own code or robots.txt shows it. The card shows how many bots are blocked or challenged at the edge.

GEOflux detects the provider in front of your site (Cloudflare, Akamai, Fastly, AWS CloudFront, Vercel, Netlify or Sucuri, or none) and classifies how each bot is treated:

• Refused by name: your site blocks the bot outright by its name.
• Blocked, cause unclear: the bot is refused, but the rule that did it could not be pinned down.
• Stopped by a security check: the bot hits a "prove you're human" challenge it cannot pass (a Managed challenge).
• Blocked by a code check: the page only appears after running browser code the bot does not run, so it sees a blank page (a JS challenge).
• Blocked by location: the bot runs from a region your site blocks (Geo-blocked).
• Let in, but throttled: the bot gets in but is Rate-limited, so large sites may not be fully read.
• Needs a closer look: something unusual that could not be classified. Open the row for the raw response.

Findings are grouped by the kind of block, and within each group ranked by impact. Blocking a user-prompted or answer engine bot is high impact, because it removes you from AI answers. Blocking a training crawler is lower urgency. Open a finding for what caused it and the edge's response as evidence.

Manual URL check

To test a specific page on demand, use Manual URL check on the Edge / WAF page: paste a URL on your own domain and click Run check. It shows how GPTBot, ChatGPT-User, PerplexityBot and Google-Extended are treated, and which provider served the page. Show raw responses gives the full detail. You can run 10 manual checks a week; the count resets every Monday at 00:00 UTC.

---

Fixing what you find

1. Start with high impact: anything blocking a user-prompted or answer engine bot.
2. Send the finding to whoever runs the site, with its How to fix and snippet. Robots.txt fixes are usually a few lines. Edge fixes usually mean allowing the named bots in your CDN or firewall settings.
3. Verify with a manual URL check once the change is live, rather than waiting a week.
4. Read the Timeline after the next audit to confirm the fix held, and pin a chart note on the day.

Then compare with Understanding sources to see which of your pages AI already cites.