Not a crawler
Applebot-Extended does not fetch pages. Disallowing it does not reduce crawling — it only withholds permission to train on content a conventional crawler already fetched.
Purpose: Training consent. Not a crawler at all. A signal withholding permission to train on content already fetched by a conventional crawler.
Not a crawler. Like Google-Extended, it withholds permission for content Applebot has already fetched to be used in training Apple's models, without affecting crawling or Apple's search features.
10.95%
of the panel names this token in robots.txt and disallows it, for the week of 2026-10-05.
How sites treat this token
| Stance | Domains | What it means |
|---|---|---|
| FullBlock | 92 | Every path is disallowed for this token. |
| PartialBlock | 17 | Some paths are disallowed for this token; others are not. |
| Allowed | 12 | The token is named and explicitly allowed. |
| Unmentioned | 874 | The file never names this token. This is not consent — the site may still close it off with a blanket rule for every crawler. |
Applebot-Extended's three-year history
A site in the historical panel first named and disallowed this token in 2024-08; its most recent reading is 12.20%.