user-agent: *
disallow: 404error.html


User-agent: Slurp
Crawl-delay: 5