Re: [cobalt-users] Load Balancing / Multiplexing / Redundancy of Cobalt Servers. HELP!

[Jeff Lasman <jblists@xxxxxxxxxxxxx>]

Gregory,

Please don't write in html.

More below, interleaved...

> "Gregory C. Kock" wrote:

> I want
> all users to have the test.com domain name, so I would want to expand
> that
> way, each time a server gets full, just add another one, kind of like
> piggy
> backing.  I know you can do a cluster of servers, but all those
> servers
> would be the mirror of one another, and not like what I had in mind.
> Let me
> try to discribe it better.  I want raq#1 which is test.com to have
> users
> 1,2,3,4,5 then let's say that the raq#1 get's filled with those users,
> I
> want to add raq#2, which will also be test.com and put users
> 6,7,8,9,10 etc
> etc.  Is this possible at all?

The Sunday School teacher said "With God, all things are possible." So I
raised my hand and asked, "Yeah?  Ever try striking a match on a wet
cake of soap?"  I was eleven.

But I was wrong.  As I know now, there are chemicals available, with
which one could coat a match head, that burst into flame when touched by
a wet cake of soap.

And so you could do this.  Even with a RaQ2.

Although not easily, since a RaQs software is NOT designed for this kind
of use.  The RaQ series is for hosting multiple domains.

If you really don't want to make toast, why buy a toaster?  That's the
entire concept behind an appliance, isn't it?

That said, let's answer the question.

First, a bit of background.

Most of you know me as a Cobalt-line consultant specializing in RaQs,
but we wear a few other hats as well.  We also own Mailtraq North
America (http://www.mailtraqna.com/";, the western-hemisphere distributor
of a major-line mailserver product designed for NT (flames accepted, but
only by private email; not on the list, please).

We get this question all the time.

If you do it right, you can handle thousands of users on a server.  You
need lots of memory, and at least one (preferably more) extremely fast
processors.  And a very fast hard disk, or better yet, scsi RAID array.

You also need a very good POP client.  One that hashes user lookups in a
database, NOT sequentially in a table like /etc/passwd.  Which means you
don't want the POP daemon included by default in the RaQ and in other
linux distributions.

If you have more users than one server can handle, even one extremely
fast server (a high-speed IBM mini running linux comes to mind), then
you can still do it.

There are several ways to do it.  One way is to run a preprocessor to
answer your port 25 requests, instead of sendmail.  That preprocessor
should have an awful lot of memory (probably at least a gigabyte) and
run a database such as postgreSQL or MySQL (or even Oracle if you need
the handholding and are running unix, or MS SQL if you're running NT or
W2K).  That system can do lookups (hopefully in memory) and figure out
which server really handles the user in question and forwards the email
there.  It should do this WITHOUT reading the email body; it should do
it entirely from the information in the envelope, and should pass on the
envelope and body intact, only adding an extra "Received:" header to the
body.

Another way is to use round-robin DNS to send the incoming mail to a
server at random, and to use a high-speed client-server database running
on a local area network to determine where/how to store the incoming
mail.

Then a special pop client can run on any system, also assigned from
round-robin DNS, to read the same database to retrieve the email.

This can't really be done using NFS for sharing; the locking issues will
bring any local network to its knees, if it works at all.

Lots of people are doing it; one that comes to mind is Hotmail
("http://www.hotmail.com/"; if you didn't already know <smile>). 
Microsoft tried and failed to convert Hotmail to NT.  They now run it on
BSD Unix (FreeBSD I believe, but it could be BSDI) running a heavily
modifed qmail (I think) instead of sendmail.

We (Mailtraq) have some education customers running thousands of users
on one NT system, and sendmail on a RaQ can do the same.  But above
thousands there are some real issues to overcome.

We (nobaloney.net) specialize in high-availability stuff, but you have
to realize it's NOT cheap.  Nor is it meant to be.

Hope this helps <smile>.

Jeff
-- 
Jeff Lasman <jblists@xxxxxxxxxxxxx>
nobaloney.net
P. O. Box 52672
Riverside, CA  92517
voice: (909) 787-8589  *  fax: (909) 782-0205