novellum
Novellum crawler, type: ai training.
- 211sites name it in robots.txt
- 84block it fully (39.8%)
- 73block some paths
- 54allow it explicitly
About this crawler
Crawler for search indexing, URL discovery and page rendering; a web search result says it fetches site content for customer-built agents.
- User-Agent
Novellum- robots.txt token
novellum- Respects robots.txt
- unknown
- Source (third party, not confirmed by the operator)
- https://botcrawl.com/bots/novellum-ai-crawl/
- Third-party source (Bot Crawl); operator Novellum does not confirm; identity unverified, medium confidence.
- Source says robots.txt behavior is currently unconfirmed.
- No IP information given; the user-agent token can be spoofed.
Including sites that block it only through the * group, 4.19% of all sites with a robots.txt keep it out.