User-agent: * # this directs to all spiders who access this file Disallow: /data/ Disallow: /aclogin/ Disallow: /admin/