[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [cobalt-users] Stop directory browsing
- Subject: Re: [cobalt-users] Stop directory browsing
- From: baltimoremd@xxxxxxxxxxxxxxx
- Date: Thu Apr 19 21:52:02 2001
- List-id: Mailing list for users to share thoughts on Cobalt products. <cobalt-users.list.cobalt.com>
On Fri, 20 Apr 2001, Dave Farthing wrote:
> directory they're held in. Is there any way to prevent this? Thought of
> making sure there is a index.html file in every directory but am not sure if
> that will be sufficient
You can make use of a robots.txt file to prevent a lot of robot browsing.
See http://www.robotstxt.org/wc/norobots.html for details.
Thom
././././././././././././././././././././././././././././././././././././././
baltimoremd@xxxxxxxxxxxxxxx Thom LaCosta K3HRN Webmaster