AgregatoBot
How our crawler works
Deep Scan analyses a store's public page to infer its platform, theme and apps. Publishing this method is part of the product: a market-intelligence tool that hides how it collects does not deserve its users' trust.
User-Agent
AgregatoBot/1.0 (+https://agregato.vercel.app/bot; contact@agregato.io)
- It reads only a store's public page, never a password-protected area.
- It reads robots.txt before every scan and stops when access is refused.
- It identifies itself by User-Agent and leaves a reachable contact address.
- It collects no personal data: no customer, no order, no address.
- It opens one request at a time, putting no load on the scanned server.
- Any takedown request is honoured within 48 hours, no argument.
To block AgregatoBot, add a Disallow rule for AgregatoBot in your robots.txt: it is read and honoured before every scan.