User-Agent: * Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /MarathonGuide/news/stories Disallow: /news/stories Disallow: /avserve Disallow: /javascript #Yahoo! Slurp User-agent: Slurp Disallow: /results/browse.cfm Disallow: /races/MoreReviews.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /avserve Disallow: /javascript Crawl-delay: 2 #EntireWeb (http://www.entireweb.com/about/search_tech/speedy_spider/) User-agent: Speedy Disallow: /results/browse.cfm Disallow: /races/MoreReviews.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /avserve Disallow: /javascript Crawl-delay: 2 #MJ12 http://www.majestic12.co.uk/projects/dsearch/mj12bot.php User-agent: MJ12bot Disallow: /results/browse.cfm Disallow: /races/MoreReviews.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /javascript Crawl-delay: 2 #Charlotte http://www.searchme.com/support/ - 12/13/07 User-agent: Charlotte Disallow: /results/browse.cfm Disallow: /races/MoreReviews.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /javascript Crawl-delay: 2 User-agent: msnbot Disallow: /races/MoreReviews.cfm Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /javascript Crawl-delay: 3 #Gigablast http://www.gigablast.com/spider.html User-agent: gigabot Disallow: /races/MoreReviews.cfm Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /javascript Crawl-delay: 4 #Wayback Machine User-agent: ia_archiver Disallow: /races Disallow: /bulletinboards Disallow: /results Disallow: /news Disallow: /favoriteruns Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /avserve Disallow: /javascript #Archive-It User-agent: archive.org_bot Disallow: /results/browse.cfm Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /javascript #Google User-agent: Googlebot Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /races/MoreReviews.cfm Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /MarathonGuide/news/stories Disallow: /news/stories Disallow: /javascript #Ask.com User-agent: Teoma Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /javascript User-Agent: OmniExplorer_Bot Disallow: / #Exabot User-agent: Exabot Disallow: /results/browse.cfm Disallow: /races/MoreReviews.cfm Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /javascript Crawl-delay: 6 #IRL Crawler - Texas A&M User-agent: IRLbot Disallow: /results/browse.cfm Disallow: /races/reviewsreply.cfm Disallow: /races/reviewsReply.cfm Disallow: /races/racereview.cfm Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /javascript #Shopping Sites #YahooSeeker - http://help.yahoo.com/help/us/shop/merchant/ User-agent: YahooSeeker Disallow: /races Disallow: /bulletinboards Disallow: /results Disallow: /news Disallow: /favoriteruns Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /javascript #Become-bot - http://www.become.com/site_owners.html User-agent: BecomeBot Disallow: /bulletinboards Disallow: /results Disallow: /news Disallow: /favoriteruns Disallow: /races Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /javascript #ShopWiki.com - http://www.shopwiki.com/wiki/Help:Bot User-agent: ShopWiki Disallow: /bulletinboards Disallow: /results Disallow: /news Disallow: /races Disallow: /favoriteruns Disallow: /contactus Disallow: /tellpeople Disallow: /redirect.cfm Disallow: /Redirect.cfm Disallow: /javascript