What it is
TallyhouseIndexBot is the crawler behind Tallyhouse, an index of how many of the most popular websites tell AI crawlers to stay out. It identifies itself as TallyhouseIndexBot/1.0 (+https://allanderek.github.io/tallyhouse/about/crawler/) and it does not pretend to be anything else.
What it fetches
It requests exactly one file from your site: /robots.txt. It never requests any other page, never follows links, and never downloads images, scripts or stylesheets. If the apex domain does not answer it tries the www. host, and if HTTPS does not answer it tries HTTP. That is the whole of its behaviour.
It fetches once per week, at most three attempts with backoff if a request fails, and reads at most 512 KB of the file. Across the whole panel of 1000 sites it holds no more than eight connections open at a time, so the load on any one site is a single small request a week.
What it will not do
It does not attempt to look like a browser. It sends its own user-agent, makes no effort to match a browser's TLS or HTTP fingerprint, and does not run JavaScript.
It never tries to get past an anti-automation challenge. If a site answers with a challenge instead of the file, Tallyhouse records that it could not read the policy and moves on. Those sites are published as an "unreadable" figure rather than guessed at, because a site that would not show us its robots.txt has not told us anything about its policy either way.
How to verify it is us
This crawler is registered under the token TallyhouseIndexBot, and every request it makes carries the user-agent TallyhouseIndexBot/1.0 (+https://allanderek.github.io/tallyhouse/about/crawler/).
All requests originate from the following addresses:
149.102.158.121/32
The same list is published in machine-readable form at ips.json.
A request claiming this user-agent from any other address is not ours.
Asking it to stop
Under RFC 9309 a crawler is always allowed to fetch /robots.txt itself — otherwise no crawler could ever learn the rules — so a Disallow rule cannot be the way to turn this one off. If you would rather your site were not in the panel, open an issue at the project repository and it will be removed. Each removal is recorded in data/panel/removals.json with its reason and the week it takes effect from. The date matters: the panel is the denominator of every published figure, so a removal applies from a future week rather than reaching back and quietly restating numbers that have already been published. From that week the site is no longer requested at all, and it leaves the denominator rather than counting as a week we failed to read it.
Why it exists
Tallyhouse publishes what sites ask of AI crawlers, weekly, from a committed ledger that anyone can re-derive. Reading robots.txt is the only way to learn what a site asks, and asking is the only thing this crawler does.