googleother
Google crawler, type: ai training.
- 1,068,885sites name it in robots.txt
- 99,294block it fully (9.3%)
- 912,089block some paths
- 57,502allow it explicitly
About this crawler
Generic crawler used by various Google product teams to fetch publicly accessible content, e.g. one-off crawls for internal research and development.
- User-Agent
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; GoogleOther)- robots.txt token
googleother- Respects robots.txt
- yes (third party: source)
- Operator page
- https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers
- Second published UA: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GoogleOther) Chrome/W.X.Y.Z Safari/537.36
- Page gives user-agent GoogleOther robots.txt example; IP list URL not captured from fetch.
- W.X.Y.Z is the page's Chrome version placeholder.
Including sites that block it only through the * group, 4.27% of all sites with a robots.txt keep it out.