[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [cobalt-users] [RaQ2] combined-access files
- Subject: Re: [cobalt-users] [RaQ2] combined-access files
- From: "Gerald Waugh" <gerald@xxxxxxxxx>
- Date: Thu Jul 6 04:54:49 2000
Thanks Dylan and Grahm
I thought it was a web spyder but I didn't understand the slurp@xxxxxxxxxxx
and http://www.inktomi.com There were entries in there yesterday and today.
It's a users site.
Gerald
> Why is this user looking for robots.txt?
>
> www.koether.com 211.169.241.21 - - [04/Jul/2000:03:21:47 -0400] "GET
>/robots.txt
> HTTP/1.0" 404 836 "-" "Slurp/si (slurp@xxxxxxxxxxx;
>http://www.inktomi.com/slur