# We don't block any particular robot software in robots.txt, as we # judge them by behavior rather than name. Abusive IPs which make # hundreds of requests in a short period of time will be IP banned. User-agent: * Disallow: