Re: [cobalt-users] Email filters that check to see if address is real

["Rod Poplarchick" <rod@xxxxxxxxxx>]

----- Original Message -----
From: "Rod Poplarchick" <rod@xxxxxxxxxx>
To: <cobalt-users@xxxxxxxxxxxxxxx>
Sent: Tuesday, April 02, 2002 10:11 PM
Subject: [cobalt-users] Email filters that check to see if address is real


> I received and email filter check from a server that host an email address
I
> was trying to send to.
>
They email me back
<snip>
On Tue, Apr 02, 2002 at 10:05:52PM -0800, Rod Poplarchick wrote:
> I would like to know the filter you are using on your server that checks
to
> see if this is a real email address.
>
> I am running a Cobalt Raq3 and would like to install something like this
on
> my server.

We couldn't find anything that gave us what we needed, so we are
writting it ourselves.  It is called flod (first line of
defense), and we intend to GPL it when complete.

It makes no decisions about delivery or rejection until it has
received the entire envelope...  thus, each recipient can have
their own configuration for what mail to accept or decline.

The base philosophy is to reject as much as possible in the
envelope (rather than after receipt) to save bandwidth.

DNS based tests, blacklists and whitelists, can all cause hard
rejections, soft rejections, warning header insertion, or simply
be ignored... a unique config file for each user decides. (with
defaults for those without configs) When it's complete, we'll
work on web page management of the config files.

When mail is received from a previously unknown address, a probe
is sent to the envelope from, and the mail is soft rejected for a
time... to allow time for the probe to bounce... or for the
sender to attempt to send mail to too many spam-trap addresses
This maintains a list of addresses which cannot receive email,
which can be used as a blacklist by the config files.  And a list
of addresses that attempt to mail to too many spam-trap
addresses, another potential blacklist.

If we decide to reject mail from a valid address because of some
blacklist, we send them a notice explaining why we are rejecting
their mail, and offering to add their from address to a recipient
specific whitelist, automatically, if they simply reply to the
notice.  This is based on the theory that if spammers had real
email addresses, they'd get mailbombed anyway, so they don't.
Besides, it would be unmanageable for a spammer to whitelist
himself in this way for every potential recipient, and the act of
doing so would likely leave an audit trail straight to him.

All rejections are logged to a global log file, and a log file
specific for each recipeint.  The latter is intended to be useful
when we get the web pages up, for users to see what mail is being
rejected and why, to help in deciding what blacklists to use, and
who to add to their personal black and whitelists. These have
alreadly proven invaluable to support staff.

The front end is written in C, and most of the logic is written
in shell scripts.  I intend to port the parts that stablize to C,
and much of the remainder may be ported to perl... but spam
blocking, like all security, is a "cat and mouse" game, so we
need the flexibility that scripting allows for... just now, for
instance I'm blocking a mass spamming from AOL servers.... we
can't block the servers, because they are legit AOL servers, and
that would be a support nightmare...  We notice though, that they
all claim to be from webmail netscape addresses, many of which
AOL seems to beleive are legit, so they get past the probe
tests...  however, they all contain certain identifying anomalies
in their headers... so when all the filters fail, and we attempt
to deliver, we look at the headers, and if it's one of these, we
reject it, and move the sender to a blacklist, so that subsequent
mail from that address will reject in the envelope, without
having to receive the mail.

It's also renaming mime file attatchments with dos executable
names.

There's still some work to do... a few sites, for instance,
reject our probes due to either the length of the from address,
or the characters it contains. (it is a GPG encryption of the
data we need to blacklist on a bounce, or whitelist on a reply)
we may have to use something simpler.

I'm quite pleased with the results so far though, the automatic
whitelisting is the most impressive feature, as it allows for
more stringent spam blocking...  but it doesn't work with all
mailing lists, more work to do...

  -smj
</snip>