Rattlesnakes By Mail

Daily snapshot 2026-11-15

2026-11-15 UTC

The numbers come from one website. They describe what these crawlers did here, not across the web.

Period: 2026-11-15T00:00:00Z to 2026-11-16T00:00:00Z (00:00Z to 24:00Z). Generated 2026-11-16T03:18:38.827Z. Schema version 1. Licence CC BY 4.0. Cite as: Benes, Peter. Rattlesnakes By Mail: A Field Guide to the Crawlers of the Web. benesthemenace.com. Method: https://rattlesnakesbymail.com/method

Requests by crawler on 2026-11-15Requests on 2026-11-15 by crawler, against each crawler's 28-day median. Googlebot: 98 requests, 28-day median 37; Bingbot: 74 requests, 28-day median 33; ClaudeBot: 48 requests, 28-day median 49; GPTBot: 27 requests, 28-day median 4; OAI-SearchBot: 24 requests, 28-day median 6; ChatGPT-User: 20 requests, 28-day median 0; Claude-User: 19 requests, 28-day median 0; PerplexityBot: 18 requests, 28-day median 0; Applebot: 16 requests, 28-day median 1; Perplexity-User: 16 requests, 28-day median 0; Claude-SearchBot: 15 requests, 28-day median 0; OAI-AdsBot: 12 requests, 28-day median 0.GooglebotGooglebot: 98 requests28-day median 3798BingbotBingbot: 74 requests28-day median 3374ClaudeBotClaudeBot: 48 requests28-day median 4948GPTBotGPTBot: 27 requests28-day median 427OAI-SearchBotOAI-SearchBot: 24 requests28-day median 624ChatGPT-UserChatGPT-User: 20 requests20Claude-UserClaude-User: 19 requests19PerplexityBotPerplexityBot: 18 requests18ApplebotApplebot: 16 requests28-day median 116Perplexity-UserPerplexity-User: 16 requests16Claude-SearchBotClaude-SearchBot: 15 requests15OAI-AdsBotOAI-AdsBot: 12 requests12
Bars show requests on 2026-11-15. The black tick on each row marks that crawler's 28-day median.

Facts

This selection has expired; it is listed in snapshot.json.

Crawlers

CrawlerRequestsVerified shareDistinct paths28-day medianSweep dayFirst pathrobots.txtsitemap
Googlebot9864.3%7737no/robots.txt162
Bingbot7473%5933no/robots.txt11
ClaudeBot4870.8%1749no/robots.txt926
GPTBot2759.3%274no/sitemap-index.xml03
OAI-SearchBot2420.8%216no/robots.txt40
ChatGPT-User200%200no/userfiles00
Claude-User190%190no/telescope/requests00
PerplexityBot180%180no/gcp-credentials.json00
Applebot166.3%161no/userfiles/x10
Perplexity-User160%160no/workspace/.env00
Claude-SearchBot150%150no/z9x8c7v6b5-debug-trigger-rattlesnakesbymail.com00
OAI-AdsBot120%120no/api%2F.env00

MCP calls

ClientToolCalls
BrickBlueBotlookup_crawler1
INITE Atlas Verifierlist_changes1
mcpask1
mcpsubmit_note1
mcp-index__mcp_index_nonexistent__1
rokmcp-probelist_changes1
verifymcp-probe__verifymcp_auth_probe_7b48b73f4adb1e7e__1

Snapshot pickup

For each of the previous seven snapshots, the first fetch by each verified crawler, in minutes after publication, and the c tag on that fetch.

SnapshotCrawlerMinutes after publicationTag
2026-11-14OAI-SearchBot77none
2026-11-14GPTBot77h
2026-11-14Bingbot293none
2026-11-14Googlebot1,135none
2026-11-13OAI-SearchBot78none
2026-11-13GPTBot78h
2026-11-13Bingbot293none
2026-11-13Googlebot628none
2026-11-13GoogleOther881none
2026-11-12Bingbot292none
2026-11-12Googlebot840none
2026-11-12OAI-SearchBot1,517none
2026-11-12GPTBot1,517p
2026-11-11Bingbot294none
2026-11-11Googlebot1,532none
2026-11-11GoogleOther1,670h
2026-11-11OAI-SearchBot2,958none
2026-11-11GPTBot2,958p
2026-11-10Bingbot294none
2026-11-10ClaudeBot1,178none
2026-11-10Googlebot2,934none
2026-11-10GoogleOther3,416none
2026-11-10OAI-SearchBot4,398none
2026-11-10GPTBot4,398p
2026-11-09Bingbot295none
2026-11-09Googlebot568none
2026-11-09GoogleOther568f
2026-11-09ClaudeBot2,670none
2026-11-09OAI-SearchBot5,838none
2026-11-09GPTBot5,838p
2026-11-08Bingbot297none
2026-11-08Googlebot1,977none
2026-11-08GoogleOther1,987f
2026-11-08ClaudeBot4,161none
2026-11-08OAI-SearchBot7,278none
2026-11-08GPTBot7,278p

Open questions

If you can answer one of these with a source, send a note: POST https://rattlesnakesbymail.com/notes with target question:<id>, where <id> is the number shown with the question, and body=<your answer and its source>.

Ask for something this snapshot lacks

What data about web crawlers did you look for and not find? Search https://rattlesnakesbymail.com/search?q=your+question. Searches with no match are recorded as gaps. A gap is published at /questions only when an editor publishes it; nothing is published automatically. To suggest something this daily snapshot should include, send a note with target snapshot:261115.

To send it, POST https://rattlesnakesbymail.com/notes as a web form or as JSON with target=snapshot:261115 and body=<your note>. The same note can be sent with target_type=snapshot and target_id=261115. The optional author_claim field says who is writing: a person, a model or an agent.

Links

Next snapshot: 2026-11-16