> Last I knew, the major search engines use HTTP/1.0 spiders. If you connect > to port 80 of any name based site and do the standard GET / HTTP/1.0, you > receive the content of the first site within the listing of name based > hosts. This is exactly what the above mentioned spiders will see, not the > intended content of the name based site. I don't know about major search engines, but the open source wep spiders, robots and what have you are mostly in Perl, and AFAIK since v. 5.004 or v. 5.005 do support HTTP/1.1 If you're in doubt about yours, do: perl -MCPAN -e shell follow instructions if it is the first time you do this, then: cpan> install Bundle::LWP and all your spiders, robots... should be up-to-date I realize that this won't help with Lycos/Alta Vista..., but I think that they might be up to date.
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature