[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[cobalt-users] [RaQ2] combined-access files
- Subject: [cobalt-users] [RaQ2] combined-access files
- From: "Gerald Waugh" <gerald@xxxxxxxxx>
- Date: Wed Jul 5 16:25:25 2000
Sorry for the amount of included data,
but the "these items in my combined-access log files looked strange/
Why is this user looking for robots.txt?
www.koether.com 211.169.241.21 - - [04/Jul/2000:03:21:47 -0400] "GET /robots.txt
HTTP/1.0" 404 836 "-" "Slurp/si (slurp@xxxxxxxxxxx; http://www.inktomi.com/slur
p.html)"
www.koether.com 211.169.241.21 - - [04/Jul/2000:03:21:57 -0400] "GET / HTTP/1.0"
200 1012 "-" "Slurp/si (slurp@xxxxxxxxxxx; http://www.inktomi.com/slurp.html)"
www.koether.com 216.35.103.61 - - [04/Jul/2000:09:59:41 -0400] "GET /robots.txt
HTTP/1.0" 404 836 "-" "Slurp/si (slurp@xxxxxxxxxxx; http://www.inktomi.com/slurp
.html)"
www.koether.com 216.35.103.61 - - [04/Jul/2000:09:59:42 -0400] "GET / HTTP/1.0"
200 1012 "-" "Slurp/si (slurp@xxxxxxxxxxx; http://www.inktomi.com/slurp.html)"
www.koether.com 209.67.206.133 - - [04/Jul/2000:16:44:06 -0400] "GET /robots.txt
HTTP/1.0" 404 836 "-" "Slurp/si (slurp@xxxxxxxxxxx; http://www.inktomi.com/slur
p.html)"
www.koether.com 209.67.206.133 - - [04/Jul/2000:16:44:07 -0400] "GET / HTTP/1.0"
200 1012 "-" "Slurp/si (slurp@xxxxxxxxxxx; http://www.inktomi.com/slurp.html)"
www.vma332.com 205.188.208.41 - - [04/Jul/2000:19:49:23 -0400] "GET / HTTP/1.0"
200 5556 "-" "Mozilla/4.0 (compatible; MSIE 5.0; MSNIA; AOL 5.0; Windows 98; Dig
www.koether.com 216.35.116.90 - - [05/Jul/2000:07:52:19 -0400] "GET /robots.txt
HTTP/1.0" 404 836 "-" "Slurp/si (slurp@xxxxxxxxxxx; http://www.inktomi.com/slurp
.html)"