phindbot
Phind crawler, type: ai search.
- 41,000sites name it in robots.txt
- 32,482block it fully (79.2%)
- 2,960block some paths
- 5,558allow it explicitly
About this crawler
AI answer engine for developers; makes isolated, on-demand requests for real-time web search rather than crawling sites.
- User-Agent
PhindBot- robots.txt token
phindbot- Respects robots.txt
- yes
- Source (third party, not confirmed by the operator)
- https://knownagents.com/agents/phindbot
- Third-party source (Known Agents, formerly Dark Visitors); operator Phind does not confirm.
- Source says 'expected to follow robots.txt rules'.
- Category AI Assistant, country United States.
Including sites that block it only through the * group, 4.22% of all sites with a robots.txt keep it out.