Thanks for all the advice. It's been extremely helpful.
RE: the comment for local caching name server - I'd not really thought
about that when I was deploying these, but it makes sense and I rolled
that out this afternoon.

RE: RulesDuJour
I didn't find these things documented anywhere. Ie. What's for
production, what's for research, when not to mix-n-match, why one is
depreciated for another, etc.
As I said before - I was trying them by trial and error to see what
works while tracking my timing... At the end of the day I'm left w/ a
much edited and picked apart parameter list for 'TRUSTED RULESETS'.

I had been on the SURBL site just this morning but nothing really
'clicked' for me. I re-read the docs, I knew it already existed in
/usr/share/spamassassin, etc.

I went over to William Stearn's website as well thinking I'd just had
a duffer file or something and saw that the last update was July 3rd -
and just assumed that I was meant to be using it. I mean, it's
integrated into the RDJ's, the site's updated regularly, he seems like
a pretty legit player, etc. What's a girl to do?

In any case - I've updated all local documentation for the next
person, the next time around. Many thanks!

-Peter Farrell

On 03/07/07, Matt Kettler wrote:
> Peter Farrell wrote:
> > Hi all.
> >
> > Testing new setup:
> > CentOS 4.4
> > amavisd-new-2.5.1
> > SpamAssassin version 3.2.1
> > running on Perl version 5.8.5
> > +RulesDuJour
> > Quad proc Dell PE w/ 4 GB RAM.
> >

> Point blank. In general, *NOBODY* should use WS's blacklist file's for
> ANYTHING. It is most unfortunate that RDJ has a built-in configuration
> for this file.
> Just take a look at the size of the files. sa-blacklist is over 24 MB!
> 1) the uri blacklist is redundant with SURBL. SURBL is lightweight and
> reasonably fast, while the uri blacklist is a heavy memory burden and
> relatively slow.
> 2) the email address blacklist is interesting for research purposes, but
> it's real-world use is almost pointless. spammers rotate domains in from
> addresses so often that the gains of this blacklist are limited, and the
> memory consumption is absurd.
> The files add something like 500MB to an instance of SA. That's *HUGE*.
> Check your memory usage and see if the blacklist file is making your box
> page. your box *might* be enough to handle the sa-blacklist, but
> personally I'd consider your box kinda borderline stats-wise for running
> sa-blacklist. I'd generally think more on the scale of 8GB of ram unless
> I was going to constrain SA to only existing in 1 or 2 instances.
> > So my questions are:
> > 1. is the timing 'normal' when using the blacklist rules called
> > through 'spamassassin'? Is it just a storm in a teacup? When it's
> > called from Perl will it all be loaded into memory and the timing will
> > drop down?

> Well, calling 'spamassassin' with sa-blacklist loaded is going to be
> very painful. sa-blacklist will cause SA to initialize around 500MB of
> memory, that's not quick.
> Or were those multi-minute times from amavis? That would be a bit much,
> and I'd be checking to see if you're thrashing your swap partition.
> Even so, I'd still expect it to take a least 60 seconds to scan a
> message with these blacklist files loaded, on a very fast CPU.
> > 2. are the rules compatible w/ the 3.2 branch of SA?

> Yes, both of WS's blacklist files are technically compatible with most
> any version of SA, save very, very old ones that don't support the uri
> keyword. (at the very least, both will work with anything from 2.40 and
> higher. digging back futher than 2.40 is an archaeological dig I'm not
> really interested in at the moment).
> However, in practice, sa-blacklist is not practical for real-world use,
> so you could also say it's incompatible with every version of SA.
> > 3. if it's 'wrong' how does one debug further? I've enabled level 5 in
> > amavisd.conf & 'smtpd -v' at the top of my Am I looking in
> > the wrong place? Am I missing some sort of Perl module that would
> > mitigate this in some way? (I'll list these at the end)

> Nope. sa-blacklist is just too huge for practical purposes. SA is
> designed to efficiently support hundreds, even thousands of
> blacklist_from's, but sa-blacklist has hundreds of thousands of them.
> (691,372 in fact).