Re: [cobalt-users] Robots.txt

[Elmer Fuddpucker <elmer@xxxxxxxxxxxxxx>]

> User-agent: *
> Disallow: /members
> /images
> /somefolder
> /someotherfolder

	Just to raise the bar a bit. Leaving the trailing slash off
in a robots.txt file will cause /images, images.html, images.gif to
be ignored. Adding the trailing slash would result in only the
directory /images/ to be ignored. I'm just pointing this out because
I, who does a great deal of search engine related work, and who
knows all this stuff have still been bitten by this a couple of
times while working sites which were designed by someone else and
which, for example, contain a etc.html page and an etc directory.

	brent

	http://www.churchindenver.org/