Re: [cobalt-users] Robots.txt

["Carrie Bartkowiak" <ravencarrie@xxxxxxxx>]

> Just to raise the bar a bit. Leaving the trailing slash off
> in a robots.txt file will cause /images, images.html, images.gif to
> be ignored. Adding the trailing slash would result in only the
> directory /images/ to be ignored.

Thanks, Brent.
I noticed this difference later last night when I was looking at my
own robots.txt file (mine have all of the slashes at the end) and was
wondering what the differences might be, if any, because I remembered
that I hadn't typed it in the post with the ending slash.
Now I know - thank you!

Any more gems re: search engines you'd like to share? I had 113 visits
to the server from different crawlers yesterday, and I think more tips
like that would be really appreciated by my clients. I know I'd
appreciate them.  ;)

*IF* you've got time to send that through, send it off-list so that
it's not dreaded OT mail - or sending it through Thom's hosting list
would be good.

CarrieB