opencrawlers

tiktokspider

ByteDance crawler, type: ai training.

About this crawler

ByteDance crawler that fetches content from websites when users share external URLs on TikTok, retrieving metadata and thumbnails for link previews.

User-Agent
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; TikTokSpider; ttspider-feedback@tiktok.com)
robots.txt token
tiktokspider
Respects robots.txt
no
Source (third party, not confirmed by the operator)
https://knownagents.com/agents/tiktokspider

Including sites that block it only through the * group, 4.25% of all sites with a robots.txt keep it out.

All AI crawlers

What we collect, and how to change your mind