Hi all.

Testing new setup:
CentOS 4.4
amavisd-new-2.5.1
SpamAssassin version 3.2.1
running on Perl version 5.8.5
+RulesDuJour
Quad proc Dell PE w/ 4 GB RAM.

Using calls to the timestamp function I've been testing this setup
over the past week. While following the debug output I've removed:
SARE_SPECIFIC
SARE_FRAUD and
SARE_HEADER0 from my TRUSTED_RULESETS in RulesDuJour/config

And also removed
99_sare_fraud_post25x.cf,
70_sare_header0.cf,
70_sare_specific.cf in /etc/mail/spamassassin -D --lint. It is not
compatible with SA3.2.

Fair enough. But the processing time during the manual test was still
really slow. Depending on the message, the total processing time
averaged between 8-15 >>minutes<< per message!
*If I then dropped both the blacklist[-uri] out, the timing was a
consistent ~45 seconds per message.
(using)
# su vscan -c 'spamassassin -D < sample-spam-GTUBE-junk.txt 2>&1' |
timestamp > $HOME/SAdebug_spam-GTUBE_10

[root@sabik ~]# head -1 SAdebug_spam-GTUBE_10; tail -1 SAdebug_spam-GTUBE_10
10:03:02.508 2.354 2.354 [32673] dbg: logger: adding facilities: all
10:12:29.882 569.727 0.000

It was down to the 2 blacklist files. So I removed them. I couldn't
see it in an 'obvious' way in the debug output, it would just hang
forever after:
dbg: plugin: loading Mail::SpamAssassin::Plugin::ImageInfo from @INC
- So, I pulled all the rulesdujour out of /etc/mail/spamassassin and
added them in one by one, along w/ a debug test message until I could
find which rules were holding it back.

After, I put it in the production stream (w/ no blacklist) and let
around 5000 messages through. (With pyzor-razor2-dcc-SA-amavisd-clamd
all running correctly.) I awk'd out the timing from the mail.log I was
seeing the general average 'total processing time' between 4-7 seconds
per message. No errors in test debug output or anything via syslogd.
I'm quite happy with this, but I'd like to make use of the blacklist
as well!

So my questions are:
1. is the timing 'normal' when using the blacklist rules called
through 'spamassassin'? Is it just a storm in a teacup? When it's
called from Perl will it all be loaded into memory and the timing will
drop down?
2. are the rules compatible w/ the 3.2 branch of SA?
3. if it's 'wrong' how does one debug further? I've enabled level 5 in
amavisd.conf & 'smtpd -v' at the top of my master.cf. Am I looking in
the wrong place? Am I missing some sort of Perl module that would
mitigate this in some way? (I'll list these at the end)

-Peter Farrell
Cardiff, Wales


############################
installed modules
############################
Archive::Extract -- 0.18
Archive::Tar -- 1.30
Archive::Zip -- 1.18
BerkeleyDB -- 0.31
CPAN -- 1.9102
CPAN::Reporter -- 0.44
Class::ErrorHandler -- 0.01
Class::Loader -- 2.03
Compress::Raw::Zlib -- 2.004
Compress::Zlib -- 2.004
Config::Tiny -- 2.10
Convert::ASCII::Armour -- 1.4
Convert::PEM -- 0.07
Convert::TNEF -- 0.17
Convert::UUlib -- 1.08
Crypt::Blowfish -- 2.10
Crypt::CAST5_PP -- 1.04
Crypt::CBC -- 2.22
Crypt:ES -- 2.05
Crypt:ES_EDE3 -- 0.01
Crypt:SA -- 0.14
Crypt::IDEA -- 1.08
Crypt::OpenPGP -- 1.03
Crypt::OpenSSL::RSA -- 0.24
Crypt::OpenSSL::Random -- 0.03
Crypt::Primes -- 0.50
Crypt::RIPEMD160 -- 0.04
Crypt::RSA -- 1.58
Crypt::Random -- 1.25
Crypt::Rijndael -- 1.04
Crypt::Twofish -- 2.12
Cwd -- 3.25
DB_File -- 1.815
Data::Buffer -- 0.04
Data:ump -- 1.08
Digest::MD2 -- 2.03
Digest::MD5 -- 2.36
Digest::SHA -- 5.44
Digest::SHA1 -- 2.11
Encode:etect -- 1.00
Error -- 0.17008
ExtUtils::CBuilder -- 0.19
ExtUtils::MakeMaker -- 6.32
File::Copy::Recursive -- 0.33
File::HomeDir -- 0.65
File::Temp -- 0.18
File::Which -- 0.05
File:ushd -- 0.99
HTML::Parser -- 3.56
IO -- 1.23
IO::CaptureOutput -- 1.03
IO::Compress::Base -- 2.004
IO::Compress::Zlib -- ???
IO::Socket::INET6 -- 2.51
IO::Socket::SSL -- 1.06
IO::Stringy -- 2.110
IO::Zlib -- 1.05
IP::Country -- 2.23
IPC::Cmd -- 0.36
IPC::Run3 -- 0.037
Image::Info -- 1.24
LWP -- 5.805
Locale::Maketext::Simple -- 0.18
Log::Message -- 0.01
Log::Message::Simple -- 0.01
MIME-tools -- ???
MIME::Base64 -- 3.07
Mail -- ???
Mail:KIM -- 0.24
Mail::SPF -- v2.004
Mail::SPF::Query -- 1.999.1
Mail::SpamAssassin -- 3.002001
Math::Pari -- 2.010709
Module::Build -- 0.2808
Module::CoreList -- 2.11
Module::Load -- 0.10
Module::Load::Conditional -- 0.16
Module::Loaded -- 0.01
Module::Pluggable -- 3.6
Net -- ???
Net::CIDR::Lite -- 0.20
Net:NS -- 0.59
Net:NS::Resolver::Programmable -- 0.002.2
Net::IP -- 1.25
Net::Ident -- 1.20
Net::SSLeay -- 1.30
Net::Server -- 0.96
NetAddr::IP -- 4.004
Object::Accessor -- 0.32
Package::Constants -- 0.01
Params::Check -- 0.26
Perl -- 5.8.5
Pod::Escapes -- 1.04
Pod::Parser -- 1.35
Pod::Simple -- 3.05
Probe::Perl -- 0.01
Socket6 -- 0.19
Sort::Versions -- 1.5
Sys::Hostname::Long -- 1.4
Tee -- 0.13
Term::ReadKey -- 2.14
Term::ReadLine -- 1.01
Term::UI -- 0.14
Test::Harness -- 2.64
Test::Reporter -- 1.27
Tie::EncryptedHash -- 1.8
Time::HiRes -- 1.9707
Time::Local -- 1.17
URI -- 1.35
Unix::Syslog -- 0.99
YAML -- 0.62
razor-agents -- ???
version -- 0.7203
########################