User-agent: * Disallow: /bot-trap/ Disallow: /mike/ Disallow: /rob/gallery/ Disallow: /rob/Lava/ Disallow: /rob/images/ Disallow: /rob/em/ Disallow: /mt/ Disallow: /cgi-bin/ Disallow: /images/ Disallow: *.jpg Disallow: *.gif Request-rate: 1/5 # maximum rate is one page every 5 seconds Visit-time: 0000-0600 # only visit between 6:00 AM and 8:45 AM UT (GMT)