[cobalt-developers] Raq3 Mail Stats Inaccuracies

["Jose Luis Aguilar" <jlaguilar@xxxxxxx>]

This is just a note about some inaccuracies I found in the mail stats
generated by the Raq3. If any cobalt member is reading, I would appreciate
if you make a note of these problems for future releases.

First, this is the way Cobalt generates mail stats:

1. They take the maillog (/var/log/maillog) and convert it to web common log
format.

This is something like:

jlaguilar@xxxxxxx - - [12/May/2000:04:20:20 +0000] "GET
/jlaguilar@xxxxxxxxxxxxxxx/" 200 11064 "http://usa.net/";
"http://mail.localdomain.com/";

Where:

'jlaguilar@xxxxxxx' is the email of the person who sent the message (From
field).
'[12/May/2000:04:20:20 +0000]' is the date and time when it was sent.
'"GET /jlaguilar@xxxxxxxxxxxxxxx/"' is the address where it is supposed to
go (To Field).
'200 11064' is the size of the message (11064 bytes).
'http://usa.net/' is the domain from where the message came.
'http://mail.localdomain.com/' is the domain where the message is going.

Using this format:

  1.1 All incoming messages have the local mail server
      as the domain where the message is going to.

  1.2 All outgoing messages have the '"GET /jlaguilar@xxxxxxxxxxxxxxx/"'
      set to the remote address.

2. Then, this log is interpreted by analog (web server stats program).
Analog was meant to interpret web stats and not mail stats. Even though it
makes an ok job at interpreting this modified mail stats, they really don't
translate well.

The following problems are found using analog interpretation of the logs:

1. In the Outgoing Messages per Domain Report:

In this report I get a huge amount of mail going out to the local mail
server (this would mean users of the same local domain sending messages to
each other). Cobalt gets this number adding up the last parameter in the log
format (the domain where the message is going to). Because all incoming mail
has the local mail server as its destination (local domain), all incoming
mail gets added to the outgoing message count. To FIX this, you need to
check that the message came from the local domain.

2. In the Incoming Messages per Address Report:

Cobalt gets these stats from the '"GET /jlaguilar@xxxxxxxxxxxxxxx/"' part of
the log. This part of the log identifies the email address where the message
is supposed to go. All incoming messages have local email addresses in this
field. Outgoing messages can have either local or external addresses in this
field. Cobalt gets the Incoming Messages count from this field without
discriminating local from external addresses. Therefore if I send a message
to foo@xxxxxxxxxxxxxxx , the log would have a line like:

jlaguilar@xxxxxxxxxxxxxxx - - [12/May/2000:04:20:20 +0000] "GET
/foo@xxxxxxxxxxxxxxx/" 200 11064 "http://localdomain.com/";
"http://otherdomain.com/";

Therefore, this message and its email address (foo@xxxxxxxxxxxxxxx) get
added to my Incoming Messages per Address report, even though it is not a
local user, and it was not an incoming message. To FIX this you need to
check that the 'domain the message is going to' matches the local domain.

Just my $0.02 to have a better RaQ3.

Jose Aguilar