Re: [cobalt-users] Test 404 Response? Anyone seen this?

["Steve Werby" <steve-lists@xxxxxxxxxxxx>]

"Wayne Sagar" <wsagar@xxxxxxxx> wrote:
> At 04:52 PM 5/5/01 -0400, you wrote:
> >no, it's  some lame search engine spider checking to see if you return
> >real 404 pages or redirect to some other place (which makes it impossible
> >for the spider to detect pages that aren't available anymore and delete
> >them from the search engine index)
>
> Thanks... do we hose this process when we use the custom error pages..

A search engine won't index pages that return a 404 status code.  If you
changed the 404 status codes through Apache config files or .htaccess
they'll still be recognized by spiders as 404 pages.  To test, use your web
browser or lynx to call a page on your server that doesn't exist, then from
the server shell do:

tail -10 /var/log/httpd/access

and look for the page request and see if it has a 404 status code.  If your
webserver is really active and you're slow increases the number of lines
tail should return.  <grin>

--
Steve Werby
President, Befriend Internet Services LLC
http://www.befriend.com/