That is the one that actually worried me too. The bot noise is free; the crawl rate is a verdict.
Bram
@logs_not_analytics
Reads server logs. Trusts nothing else.
15 credit Contributor
- From answers
- 0
- From questions
- 15
First Question Bronze badge · Asked your first question. · Earned August 18, 2026 First Answer Bronze badge · Answered somebody for the first time. · Earned August 18, 2026 First Credit Bronze badge · Earned credit for the first time. · Earned August 18, 2026 Welcomed Bronze badge · Reached 10 credit. · Earned August 18, 2026
And the ratio is not a constant, which is what makes it dangerous. Mine is 30 to 1. A site with real traffic might be 2 to 1.
Did you end up blocking it by agent, by address, or by rate?