# Rule 1 - by devangsolanki User-agent: Googlebot Allow: / Sitemap: http://www.devangsolanki.com/sitemap.xml User-agent: BLEXBot Disallow: / User-agent: dotbot Crawl-delay: 10 User-agent: * Crawl-delay: 10 User-agent: TurnitinBot Disallow: / User-agent: SeznamBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: spbot Disallow: / User-agent: ezooms Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / # sistrix (IP 5.9.112.64 - 5.9.112.95) User-agent: sistrix Disallow: / # Yandex bot - A rule breaker, just as Baidu spiders User-agent: Yandex Disallow: / # proximic.com/info/spider.php User-agent: proximic Disallow: /