Re: [cobalt-users] Webalizer .pkg for the Raq3 and logrotates

[John Rood <john@xxxxxxxxx>]

On Wed, 19 Jul 2000, Tarren wrote:

> 
> Dear John,
> 
> Thanks for your posting in cobalt-users about the webalizer package that you
> have developed.
> 
> I wote a shell script which does almost exactly the same thing that your
> perl script does, however, I realised that this method is potentially flawed
> by design:
> 
> The web.log files in the /home/sites/sitexx/log directories may be subject
> to rotation at inexplicable times due to the "size" paramter in the
> logrotate scripts.
> 
> If the "split logs" program runs first, putting entries into web.log, then
> the individual site log qualifies for rotation, these entries would be
> rotated into web.log.gz.1, and the webalizer run for that site would miss
> those entries. Do you see what I mean?
> 
> I think I'm right in what I'm saying here, but perhaps you could confirm or
> deny my suspicions?
> 
> The only "safe" way I can think of is to act upon access.1.gz, split out the
> entries you want on the fly, and process them. At least "access" is rotated
> predictably every night.
> 
> I hope I make some sense here, and maybe you could comment with your
> findings too, as I am not exactly sure in which order the various "rotates"
> take place. Most people don't want to edit the "logrotate" files manually,
> and in any case, they keep getting re-written when you use the UI.
> 
> Regards,
> 
> Tarren.
> 

I took a closer look at the various rotates and at the split_log script.
Indeed, it's possible that you'll end up with a gap in the logs when using
~/logs/web.log. This is caused by the fact that split_log is ran as a pre
rotate script. split_log itself calls analog, so this works fine for
analog. After running split_log (and thus analog) which concats the access
stats the the site's ~/log/web.log, the web.log might be apt for
logrotation, depending on it's size, like you indicated.

Currently i can only come up two ways around this problem: 1) change the
split_log script, so it runs webalizer (too) on the log, before it gets
rotated. Not one of the cleanest solutions imo. Or 2) avoid ~/logs/web.log
like the plague and like you suggested, use the big access.1.gz. This is
probably the way to go. But for this to work you need to split it up again
ona per site basis, run webalizer on the split logs and eventually rotate
them (or do it directly). So this results in a stats system running
 
Although it causes some overhead (splitting the access log twice) i prefer
the latter way. But, I might be overlooking 'the one very obvious way'(tm)
to do this. Anyone can shed some light on this?

Best regards,
John

--
Hi! I'm a .signature virus! Copy me into your ~/.signature to help me spread!
--
John C. Rood
UNIX programmer/Database developer/System administrator
SFARC Networks, The Netherlands - http://www.sfarc.net