Tallyhouse

Perplexity-User

Perplexity — User-initiated

All crawlers · Agent Accessibility Index

Purpose: User-initiated. Fetches a specific page because a user asked, rather than crawling broadly. Blocking this stops people using an assistant to read the site; it does not prevent training.

Fetches a page because a Perplexity user asked for it.

7.34%

of the panel names this token in robots.txt and disallows it, for the week of 2026-10-05.

How sites treat this token

StanceDomainsWhat it means
FullBlock55Every path is disallowed for this token.
PartialBlock18Some paths are disallowed for this token; others are not.
Allowed13The token is named and explicitly allowed.
Unmentioned909The file never names this token. This is not consent — the site may still close it off with a blanket rule for every crawler.

Perplexity-User's three-year history

Perplexity-User over time, 16 quarterly periods from 2023-01 to 2026-08.Perplexity-User rose from 0.00% at 2023-01 to 8.22% at 2026-08.0%2%4%6%8%10%2023-012023-092024-052025-012025-102026-088.22%

A site in the historical panel first named and disallowed this token in 2025-04; its most recent reading is 8.22%.

Note: this chart reads Common Crawl's own 611-domain historical panel, not the panel behind the stance table above — a different population from the live index panel, so levels are not comparable between the two indices.

Perplexity's other tokens

An answer engine, operating a crawler and a user-initiated fetcher.

TokenPurposePublished rate
PerplexityBotSearch14.07%

Sites that treat these tokens differently

88 domains in the panel treat Perplexity's tokens differently from one another. The first 25 are shown below.

DomainPerplexity-UserPerplexityBot
abc.net.auUnmentionedFullBlock
academia.eduUnmentionedPartialBlock
aftonbladet.seUnmentionedFullBlock
airbnb.comUnmentionedPartialBlock
aljazeera.comUnmentionedFullBlock
amap.comUnmentionedPartialBlock
android.comUnmentionedPartialBlock
aol.comUnmentionedPartialBlock
apnews.comUnmentionedFullBlock
bidmachine.ioUnmentionedAllowed
bloomberg.comUnmentionedFullBlock
branch.ioUnmentionedAllowed
chatgpt.comUnmentionedFullBlock
cloudflare.comUnmentionedAllowed
cnbc.comUnmentionedFullBlock
cookpad.comUnmentionedFullBlock
corriere.itUnmentionedAllowed
coursera.orgUnmentionedPartialBlock
dagbladet.noUnmentionedFullBlock
dcinside.comUnmentionedFullBlock
disneyplus.comUnmentionedPartialBlock
ebay.co.ukUnmentionedFullBlock
ebay.comUnmentionedFullBlock
ebay.deUnmentionedFullBlock
elmundo.esUnmentionedFullBlock