Purpose: Search. Indexes pages so an answer engine can retrieve and cite them.
Indexes pages so Perplexity can cite them in answers.
14.07%
of the panel names this token in robots.txt and disallows it, for the week of 2026-10-05.
How sites treat this token
| Stance | Domains | What it means |
|---|---|---|
| FullBlock | 98 | Every path is disallowed for this token. |
| PartialBlock | 42 | Some paths are disallowed for this token; others are not. |
| Allowed | 31 | The token is named and explicitly allowed. |
| Unmentioned | 824 | The file never names this token. This is not consent — the site may still close it off with a blanket rule for every crawler. |
PerplexityBot's three-year history
A site in the historical panel first named and disallowed this token in 2024-02; its most recent reading is 15.65%.
Perplexity's other tokens
An answer engine, operating a crawler and a user-initiated fetcher.
| Token | Purpose | Published rate |
|---|---|---|
| Perplexity-User | User-initiated | 7.34% |
Sites that treat these tokens differently
88 domains in the panel treat Perplexity's tokens differently from one another. The first 25 are shown below.
| Domain | Perplexity-User | PerplexityBot |
|---|---|---|
| abc.net.au | Unmentioned | FullBlock |
| academia.edu | Unmentioned | PartialBlock |
| aftonbladet.se | Unmentioned | FullBlock |
| airbnb.com | Unmentioned | PartialBlock |
| aljazeera.com | Unmentioned | FullBlock |
| amap.com | Unmentioned | PartialBlock |
| android.com | Unmentioned | PartialBlock |
| aol.com | Unmentioned | PartialBlock |
| apnews.com | Unmentioned | FullBlock |
| bidmachine.io | Unmentioned | Allowed |
| bloomberg.com | Unmentioned | FullBlock |
| branch.io | Unmentioned | Allowed |
| chatgpt.com | Unmentioned | FullBlock |
| cloudflare.com | Unmentioned | Allowed |
| cnbc.com | Unmentioned | FullBlock |
| cookpad.com | Unmentioned | FullBlock |
| corriere.it | Unmentioned | Allowed |
| coursera.org | Unmentioned | PartialBlock |
| dagbladet.no | Unmentioned | FullBlock |
| dcinside.com | Unmentioned | FullBlock |
| disneyplus.com | Unmentioned | PartialBlock |
| ebay.co.uk | Unmentioned | FullBlock |
| ebay.com | Unmentioned | FullBlock |
| ebay.de | Unmentioned | FullBlock |
| elmundo.es | Unmentioned | FullBlock |