opencrawlers

novellum

Novellum crawler, type: ai training.

About this crawler

Crawler for search indexing, URL discovery and page rendering; a web search result says it fetches site content for customer-built agents.

User-Agent
Novellum
robots.txt token
novellum
Respects robots.txt
unknown
Source (third party, not confirmed by the operator)
https://botcrawl.com/bots/novellum-ai-crawl/

Including sites that block it only through the * group, 4.19% of all sites with a robots.txt keep it out.

All AI crawlers

What we collect, and how to change your mind