Re: [cobalt-users] Cobalt Users Message Board (was: Cobalt user chatroom????)

["Steve Werby" <steve-lists@xxxxxxxxxxxx>]

"Brett" <cobalt@xxxxxxxxxxxxxxxxx> wrote:
> > I've personally setup a web archive of
> > this list and I've come across at least two public web archives of this
> list
> > besides those maintained by Cobalt.
>
> I have been wanting to archive the this list on my server for easier
> searching and to keep from having to store all the messages on my pc at
> home.  Would you care to elaborate on how you set up an archive or point
me
> in the direction of some reading material.  I have not found much in my
> attempts to find info.

I use procmail to drop messages from mailing lists into their own IMAP
folder (each folder exists on the server as a single text file) and then I
typically use MhonArc to convert messages to HTML.  MHonArc can be found at
http://www.nacs.uci.edu/indiv/ehood/mhonarc.html.  After building the web
archive the first time I typically use crontab to run a mhonarc command
several times per day to update the archives.  This works great, but doesn't
make it easy to build an interface for filtering and sorting records and a
search engine interface requires indexing the web archive using something
like mnoGoSearch (btw, I'm a big fan of mnoGoSearch - see
http://www.mnogosearch.com/ ).

So I've also used the following more sophisticated method in which I insert
emails into a database (MySQL or PostgreSQL) as they are received by
sendmail.  I've only been able to handle about 98% of the messages this way
due to some problems with MIME types.  Using a PHP script I wrote which runs
as a CGI, I created a record in sendmail's aliases file to route all emails
for my mailing list user to itself (which gets sent to procmail for
processing)and to the PHP script which parses the email's headers into a
handful of fields and body and stores them into a database.  In addition to
requiring much less storage space than the first method I suggested, it's
quite easy to build an application using PHP to sort and filter records,
flag messages, track views and implement a search engine within a MySQL
database by using MySQL's new FULLTEXT search functionality (though
mnoGoSearch's database structure will probably be faster).  The difficulty
level for building an application like this is somewhat high so it's
probably not worth undertaking unless you plan on building a web archive
with search engine and other bells and whistles for a large list with many
simulaneous users.

--
Steve Werby
President, Befriend Internet Services LLC
http://www.befriend.com/