[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[cobalt-users] Robots.txt
- Subject: [cobalt-users] Robots.txt
- From: "Zarrir Junior" <zarrir@xxxxxxxxxxxx>
- Date: Tue Jun 19 10:05:44 2001
- List-id: Mailing list for users to share thoughts on Cobalt products. <cobalt-users.list.cobalt.com>
Hi,
Considering there is no record for "robots" nor "robots.txt" in the
archives, i´d like to know how to use the file robots.txt located under
/etc/admserv/html of my Cobalt. By now, it reads as:
# Prevent all robots from visiting this site:
User-agent: *
Disallow: /
How can i, for instance forbid a robot to search my pages? Considering the
directory it is located (under root), which virtual sites would it affect?
Thanks in advance!
Zarrir