tinydot bot

TinydotBot fetches a website only when someone asks tinydot to import that site into their own. it does not crawl the web, and it collects nothing for search, advertising or training.

it identifies itself as:

TinydotBot/1.0 (+https://docs.tinydot.com/bot)

what it does#

• reads robots.txt first and obeys Disallow
• waits half a second between requests to the same host
• stops at a page ceiling per import, so one request can never become an endless crawl
• reads your sitemap when you have one, which is fewer requests than walking your links

how to block it#

add this to your robots.txt:

User-agent: TinydotBot
Disallow: /

it is read before anything else is fetched, and a Disallow stops that page being requested at all.

questions#

write to hi@tinydot.com.