Crawl Activity
Rank4AI — rank4ai.co.uk
Bing crawl ran 7h ago · 05 Jun, 09:02 · runs daily 06:00
Cloudflare AI Crawl Control (14d)
Edge-level bot counts, true hits — not sampled
13,866
14 bots tracked
05-13 05-20
PerplexityBot 3,113
ChatGPT-User 2,385
Googlebot 1,759
ClaudeBot 1,375
Bytespider 986
3199
Total Bing Crawls
616
Pages Indexed (Bing)
13
AI Bots Allowed
0
AI Bots Blocked
Crawler Activity
Bing (bars, left axis) vs AI bots (lines, right axis)
AI Crawler Access
GPTBot allowed
ChatGPT-User allowed
Google-Extended allowed
GoogleOther allowed
ClaudeBot allowed
Claude-Web not_mentioned
PerplexityBot allowed
Bytespider allowed
CCBot allowed
cohere-ai allowed
Amazonbot allowed
anthropic-ai not_mentioned
FacebookBot allowed
Applebot-Extended allowed
Bingbot allowed
Live Crawler Access Test (HTTP Response)
Tests if each bot actually gets a 200 response when visiting the homepage
GPTBot
ClaudeBot
PerplexityBot
Googlebot
Bingbot
Topic Clusters (44 detected)
/research/
157
pages in cluster
/ecosystem/
1
pages in cluster
/rank-on-chatgpt/
1
pages in cluster
/who-we-help/
29
pages in cluster
/ai-seo-company/
1
pages in cluster
/contact/
1
pages in cluster
/learn/
242
pages in cluster
/how-often-should-i-analyze-ai-visibility-performance/
1
pages in cluster
/ai-visibility-seo-agency/
1
pages in cluster
/free-audit/
1
pages in cluster
/privacy/
1
pages in cluster
/chatgpt-optimisation-agency/
1
pages in cluster
/duckduckgo-visibility/
1
pages in cluster
/google-ai-optimisation/
1
pages in cluster
/resources/
1
pages in cluster
/terms/
1
pages in cluster
/sitemap/
1
pages in cluster
/ai-search-agency/
1
pages in cluster
/about/
9
pages in cluster
/blog/
128
pages in cluster
/ai-seo-agency-manchester/
1
pages in cluster
/uk/
16
pages in cluster
/ai-instructions/
1
pages in cluster
/commissioncrowd/
1
pages in cluster
/ai-search-website-design/
1
pages in cluster
/ai-seo/
41
pages in cluster
/site-structure/
1
pages in cluster
/podcast/
1
pages in cluster
/ai-search/
72
pages in cluster
/ai-seo-agency-london/
1
pages in cluster
/responsible-ai/
1
pages in cluster
/ai-seo-agency-uk/
10
pages in cluster
/ai-marketing-growth/
16
pages in cluster
/best-chatgpt-optimisation-company/
1
pages in cluster
/accessibility/
1
pages in cluster
/data/
2
pages in cluster
/chatgpt-seo-company/
1
pages in cluster
/book-a-call/
1
pages in cluster
/chatgpt-ranking-optimisation-service/
1
pages in cluster
/uk-ai-seo-company/
1
pages in cluster
/visibility-in-ai-search/
1
pages in cluster
/cookies/
1
pages in cluster
/services/
12
pages in cluster
/what-we-do/
6
pages in cluster
Terminology Consistency (H1 Analysis)
2026
x (%)
search
x (%)
rank4ai
x (%)
visibility
x (%)
business
x (%)
businesses
x (%)
chatgpt
x (%)
google
x (%)
agencies
x (%)
agency
x (%)
platforms
x (%)
does
x (%)
perplexity
x (%)
overviews
x (%)
claude
x (%)