23.97%
Denominator: 997 conclusive domains. Coverage: 99.70% of the panel.
Provenance
This is why this page exists: the exact run that produced this figure, frozen alongside it rather than left to be inferred later.
| Vintage | 2 |
|---|---|
| Methodology | agents=2;protego=0.6.2 |
| Collector | f5c2b39 |
| Computed at |
Every series value for this period
| Series | Value |
|---|---|
| Effective (named or blanket) | 26.98% |
| Blanket disallow (closed to every crawler) | 5.82% |
| Coverage | 99.70% |
| Unreadable robots.txt | 0.10% |
| AI2Bot | 5.42% |
| AI2Bot-Dolma | 4.61% |
| Amazonbot | 9.53% |
| Applebot-Extended | 10.63% |
| Brightbot | 3.41% |
| Bytespider | 14.24% |
| CCBot | 16.15% |
| ChatGLM-Spider | 2.91% |
| ChatGPT-User | 11.13% |
| Claude-SearchBot | 8.53% |
| Claude-User | 8.02% |
| Claude-Web | 8.02% |
| ClaudeBot | 14.94% |
| DeepSeekBot | 4.11% |
| Devin | 2.51% |
| Diffbot | 9.63% |
| DuckAssistBot | 6.72% |
| GPTBot | 16.35% |
| Gemini-Deep-Research | 3.01% |
| Google-CloudVertexBot | 5.01% |
| Google-Extended | 14.04% |
| Google-NotebookLM | 2.61% |
| GoogleAgent-Mariner | 2.21% |
| ImagesiftBot | 6.82% |
| MistralAI-User | 5.22% |
| OAI-SearchBot | 9.93% |
| PanguBot | 4.91% |
| Perplexity-User | 7.32% |
| PerplexityBot | 14.04% |
| PhindBot | 2.31% |
| SBIntuitionsBot | 2.81% |
| TavilyBot | 2.21% |
| Timpibot | 7.52% |
| WrtnBot | 2.21% |
| YouBot | 8.02% |
| anthropic-ai | 9.23% |
| cohere-ai | 9.73% |
| cohere-training-data-crawler | 4.91% |
| iaskspider | 2.51% |
| meta-externalagent | 11.43% |
| meta-externalfetcher | 6.52% |
| meta-webindexer | 3.21% |
| omgili | 9.33% |
| omgilibot | 7.92% |
| webzio-extended | 5.62% |
Restatements
This period has been restated. The earlier figure was never edited, only superseded — it is kept here alongside the reason it changed.
| Vintage | Value | Reason |
|---|---|---|
| 1 | 23.47% | methodology v2: agent set expanded from 16 to 45 tokens after an audit found v1 incomplete (Anthropic tracked 1 of 4 live tokens; Meta's fetcher, Cohere's training crawler and Mistral absent) |
Suggested citation
Agent Accessibility Index, 2026-09-14: 23.97% (vintage 2). https://allanderek.github.io/tallyhouse/agent-accessibility/releases/2026-09-14/index.html
The two ledgers below carry this period's rows among every other period's; anything named for this period alone is listed with them.
Downloads
Every figure published on this site can be re-derived from these files. The two ledgers below are served as the exact bytes committed to the repository, not regenerated.
| File | Rows | Size | Description |
|---|---|---|---|
| Headline ledger | 20 | 2.6 KB | Every headline figure ever published, for both indices, including superseded vintages and the reason each was restated. Served as the exact bytes committed to the repository. |
| Series ledger | 948 | 130.2 KB | Every sub-series: the per-crawler rates, effective and blanket blocking, coverage, the unreadable share and the like-for-like change. Same vintage rules as the headline ledger, same exact bytes. |