Re: [cobalt-users] HostLookups, Webalizer & Logrotate, um WOW??

[John Rood <john@xxxxxxxxx>]

*snip* (read the previous messages if you wanna know what's going on)

Hi all,

Just a few remarks regarding the last thread. First of all, like Debbie
found out, you can use incremental processing of the logs, either by using
the command line opton -p or adding the Incremental directive to
webalizer.conf. And it's also possible to have webalizer use a per site
config file, but it'll always read /etc/webalizer.conf, handy for a global
setup. Any settings in a specified config file on the commandline
overrides any settings from the global config. However, with both cobalt's
and mine webalizer package, you run the risk of getting a hole in your
stats. See my previous messages for details
(http://list.cobalt.com/pipermail/cobalt-users/2000-July/016334.html).
About webalizer handling gzipped logs, this is only a feature of the 2.00
and up releases. It assumes that when you feed it a log file ending in .gz
that it's a gzippped log and does the right magic (gunzip) before
processing the log file.

One final note, concerning resolving ip addresses. Like Steven suggested,
you can turn on reverse lookups in your apache config, but this is imo
(and a lot of other people's too) bad thing to do. This might (and will)
result in slower response times from the webserver, as high as 2x 75s
(twice the maximum dns timeout) in theory. Apache will probably timeout
sooner. Why? When a client requests a page, apache first has to resolve
it's ip address, which requires a reverse dns lookup. If the client
happens to be on a slow link (or atleast it's dns) or worse, the dns might
be down> Until apache gets an answer from the dns (or the query times
out), it doesn't serve the client with anything requested. Thats why you'd
better keep hostname lookups turned off in apache itself and use an
external program to preprocess the logs before running webalizer on them.
Webalizer 1.30 can't do this by itself, but (atleast on a raq3) you'll
find logresolve in /usr/sbin what does just that. It nicely resolves all
ip adresses (when possible) in your logs. Read the manpage first. I
myself took a different approach. Since i'm using webalizer 2.00, i don't
need logresolv. 2.00 can do this just fine by itself. On a per log basis
while generating the stats, or like i do, use it as webazolver. I
preprocess all the log files with webazolver, building a dns cache (much
like the now famous analog.dns file) and after that i run webalizer, using
that cache for the resolving. This way apache still gets to do it's normal
job, without being kept from it by slow dns servers and it doesn't really
matter when a preprocessing script takes a few seconds longer doing it's
job, since nobody'll notice. 

Best regards,
John

--
Hi! I'm a .signature virus! Copy me into your ~/.signature to help me spread!
--
John C. Rood
UNIX programmer/Database developer/System administrator
SFARC Networks, The Netherlands - http://www.sfarc.net