Perl regex crash on large HTML table under SPAMD - SpamAssassin

This is a discussion on Perl regex crash on large HTML table under SPAMD - SpamAssassin ; Hi, I'm running the following rules in my local rules for a while and it works flawlessly under RedHat EL 3 (Perl 5.8.0): rawbody TABLEOBFU / ]|"[^"]*"|'[^']*')*>( ]|"[^"]*"|'[^']*')*>)*[a-z]{1,2} ( ]|"[^"]*"|'[^']*')*>)* ]|"[^"]*"|'[^']*')*>/i Recently, we move to Apple Mac OS X 10.5 ...

+ Reply to Thread
Results 1 to 5 of 5

Thread: Perl regex crash on large HTML table under SPAMD

  1. Perl regex crash on large HTML table under SPAMD

    Hi,

    I'm running the following rules in my local rules for a while and it
    works flawlessly under RedHat EL 3 (Perl 5.8.0):

    rawbody TABLEOBFU /
    ]|"[^"]*"|'[^']*')*>(<([^>]|"[^"]*"|'[^']*')*>)*[a-z]{1,2}
    (<([^>]|"[^"]*"|'[^']*')*>)*<\/td([^>]|"[^"]*"|'[^']*')*>/i

    Recently, we move to Apple Mac OS X 10.5 server with Perl 5.8.8. We
    start getting SIGCHLD error and 0/0 scores. I've found that TABLEOBFU
    rule crash if there is a large table (~32k) inside the mail... I was
    able to reproduce it under Redhat EL 5 also (Perl 5.8.8) but HTML must
    be 2 times larger (~64k).

    I also try with the following rule I got from the mailing list. Same
    problem.

    rawbody TABLEOBFU /
    ]|"[^"]*"|'[^']*')*>(<([^>]|"[^"]*"|'[^']*')*>)*[a-z]{1,2}
    (<([^>]|"[^"]*"|'[^']*')*>)*<\/td([^>]|"[^"]*"|'[^']*')*>/i

    It seems to be something related to allocated memory for the spamd
    script...

    Anything known about that issue? Is is possible to "skip" a test
    according to mail size?

    thanks

    Dominic Germain
    ---------------------------------------------
    Administrateur réseau / Network administrator
    Sogetel
    www.sogetel.net

    mailinglists@sogetel.com


    Our test RHEL setup :
    ************************************************** ****************************************
    [21256] dbg: generic: SpamAssassin version 3.2.5
    [21256] dbg: config: score set 0 chosen.
    [21256] dbg: dns: is Net:NS::Resolver available? yes
    [21256] dbg: dns: Net:NS version: 0.63
    [21256] dbg: generic: sa-update version svn607589
    [21256] dbg: generic: using update directory: /var/lib/spamassassin/
    3.002005
    [21256] dbg: diag: perl platform: 5.008008 linux
    [21256] dbg: diag: module installed: Digest::SHA1, version 2.11
    [21256] dbg: diag: module installed: HTML::Parser, version 3.56
    [21256] dbg: diag: module installed: Net:NS, version 0.63
    [21256] dbg: diag: module installed: MIME::Base64, version 3.07
    [21256] dbg: diag: module installed: DB_File, version 1.814
    [21256] dbg: diag: module installed: Net::SMTP, version 2.29
    [21256] dbg: diag: module installed: Mail::SPF, version v2.005
    [21256] dbg: diag: module installed: Mail::SPF::Query, version 1.999001
    [21256] dbg: diag: module installed: IP::Country::Fast, version 604.001
    [21256] dbg: diag: module not installed: Razor2::Client::Agent
    ('require' failed)
    [21256] dbg: diag: module installed: Net::Ident, version 1.20
    [21256] dbg: diag: module installed: IO::Socket::INET6, version 2.54
    [21256] dbg: diag: module installed: IO::Socket::SSL, version 1.14
    [21256] dbg: diag: module installed: Compress::Zlib, version 2.012
    [21256] dbg: diag: module installed: Time::HiRes, version 1.86
    [21256] dbg: diag: module installed: Mail:omainKeys, version 1.0
    [21256] dbg: diag: module installed: Mail:KIM, version 0.32
    [21256] dbg: diag: module installed: DBI, version 1.52
    [21256] dbg: diag: module installed: Getopt::Long, version 2.35
    [21256] dbg: diag: module installed: LWP::UserAgent, version 5.814
    [21256] dbg: diag: module installed: HTTP:ate, version 5.810
    [21256] dbg: diag: module installed: Archive::Tar, version 1.38
    [21256] dbg: diag: module installed: IO::Zlib, version 1.09
    [21256] dbg: diag: module installed: Encode:etect, version 1.01


  2. Re: Perl regex crash on large HTML table under SPAMD


    Dominic Germain writes:
    > Hi,
    >
    > I'm running the following rules in my local rules for a while and it
    > works flawlessly under RedHat EL 3 (Perl 5.8.0):
    >
    > rawbody TABLEOBFU /
    > ]|"[^"]*"|'[^']*')*>(<([^>]|"[^"]*"|'[^']*')*>)*[a-z]{1,2}
    > (<([^>]|"[^"]*"|'[^']*')*>)*<\/td([^>]|"[^"]*"|'[^']*')*>/i
    >
    > Recently, we move to Apple Mac OS X 10.5 server with Perl 5.8.8. We
    > start getting SIGCHLD error and 0/0 scores. I've found that TABLEOBFU
    > rule crash if there is a large table (~32k) inside the mail... I was
    > able to reproduce it under Redhat EL 5 also (Perl 5.8.8) but HTML must
    > be 2 times larger (~64k).
    >
    > I also try with the following rule I got from the mailing list. Same
    > problem.
    >
    > rawbody TABLEOBFU /
    > ]|"[^"]*"|'[^']*')*>(<([^>]|"[^"]*"|'[^']*')*>)*[a-z]{1,2}
    > (<([^>]|"[^"]*"|'[^']*')*>)*<\/td([^>]|"[^"]*"|'[^']*')*>/i
    >
    > It seems to be something related to allocated memory for the spamd
    > script...
    >
    > Anything known about that issue? Is is possible to "skip" a test
    > according to mail size?


    The best fix would be to rewrite those rules not to use /*/, to use
    /{0,n}/ instead, and to use /(?...)/ instead of /(...)/ . You could also
    upgrade to perl 5.10, which may have this bug fixed iirc; however you
    would still be vulnerable to another side-effect of that kind of regexp,
    which is exponential runtime. unlimited quantifiers like + and * are very
    bad news in SA rules.

    --j.


  3. Re: Perl regex crash on large HTML table under SPAMD

    The Mac have 2 Gig of RAM, the HP box is running with 7 Gig .

    Dominic Germain
    ---------------------------------------------
    Administrateur réseau / Network administrator
    Sogetel
    www.sogetel.net

    mailinglists@sogetel.com



    Le 08-08-05 à 14:46, Ron Smith a écrit :

    > Hmmm. I think Perl 5.8.8 is the current release. How much ram do you
    > have on your OS X Server by the way?
    >
    > Ron Smith
    > postmaster@pmbx.net
    >
    > "Having an email problem is painful, but character-building."
    >
    > On Aug 5, 2008, at 2:20 PM, Dominic Germain wrote:
    >
    >> Hi,
    >>
    >> I'm running the following rules in my local rules for a while and
    >> it works flawlessly under RedHat EL 3 (Perl 5.8.0):
    >>
    >> rawbody TABLEOBFU /
    >> ]|"[^"]*"|'[^']*')*>(<([^>]|"[^"]*"|'[^']*')*>)*[a-z]{1,2}
    >> (<([^>]|"[^"]*"|'[^']*')*>)*<\/td([^>]|"[^"]*"|'[^']*')*>/i
    >>
    >> Recently, we move to Apple Mac OS X 10.5 server with Perl 5.8.8.
    >> We start getting SIGCHLD error and 0/0 scores. I've found that
    >> TABLEOBFU rule crash if there is a large table (~32k) inside the
    >> mail... I was able to reproduce it under Redhat EL 5 also (Perl
    >> 5.8.8) but HTML must be 2 times larger (~64k).
    >>
    >> I also try with the following rule I got from the mailing list.
    >> Same problem.
    >>
    >> rawbody TABLEOBFU /
    >> ]|"[^"]*"|'[^']*')*>(<([^>]|"[^"]*"|'[^']*')*>)*[a-z]{1,2}
    >> (<([^>]|"[^"]*"|'[^']*')*>)*<\/td([^>]|"[^"]*"|'[^']*')*>/i
    >>
    >> It seems to be something related to allocated memory for the spamd
    >> script...
    >>
    >> Anything known about that issue? Is is possible to "skip" a test
    >> according to mail size?
    >>
    >> thanks
    >>
    >> Dominic Germain

    >



  4. Re: Perl regex crash on large HTML table under SPAMD - SOLVED

    replacing "*" by "{0,30}" fixed my problem! The regex works for small
    tables without crashing spamd childs.

    IMO, upgrading to Perl 5.10 on OSX is not recommended... Apple is
    tweeking a lot of stuff and there is always chances that a future OSX
    update will screw up everything.

    many thanks!


    Dominic Germain
    ---------------------------------------------
    Administrateur réseau / Network administrator
    Sogetel
    www.sogetel.net

    mailinglists@sogetel.com



    Le 08-08-05 à 15:23, Justin Mason a écrit :

    >
    > Dominic Germain writes:
    >> Hi,
    >>
    >> I'm running the following rules in my local rules for a while and it
    >> works flawlessly under RedHat EL 3 (Perl 5.8.0):
    >>
    >> rawbody TABLEOBFU /
    >> ]|"[^"]*"|'[^']*')*>(<([^>]|"[^"]*"|'[^']*')*>)*[a-z]{1,2}
    >> (<([^>]|"[^"]*"|'[^']*')*>)*<\/td([^>]|"[^"]*"|'[^']*')*>/i
    >>
    >> Recently, we move to Apple Mac OS X 10.5 server with Perl 5.8.8. We
    >> start getting SIGCHLD error and 0/0 scores. I've found that
    >> TABLEOBFU
    >> rule crash if there is a large table (~32k) inside the mail... I was
    >> able to reproduce it under Redhat EL 5 also (Perl 5.8.8) but HTML
    >> must
    >> be 2 times larger (~64k).
    >>
    >> I also try with the following rule I got from the mailing list. Same
    >> problem.
    >>
    >> rawbody TABLEOBFU /
    >> ]|"[^"]*"|'[^']*')*>(<([^>]|"[^"]*"|'[^']*')*>)*[a-z]{1,2}
    >> (<([^>]|"[^"]*"|'[^']*')*>)*<\/td([^>]|"[^"]*"|'[^']*')*>/i
    >>
    >> It seems to be something related to allocated memory for the spamd
    >> script...
    >>
    >> Anything known about that issue? Is is possible to "skip" a test
    >> according to mail size?

    >
    > The best fix would be to rewrite those rules not to use /*/, to use
    > /{0,n}/ instead, and to use /(?...)/ instead of /(...)/ . You
    > could also
    > upgrade to perl 5.10, which may have this bug fixed iirc; however you
    > would still be vulnerable to another side-effect of that kind of
    > regexp,
    > which is exponential runtime. unlimited quantifiers like + and * are
    > very
    > bad news in SA rules.
    >
    > --j.



  5. Re: Perl regex crash on large HTML table under SPAMD - SOLVED


    Dominic Germain writes:
    > replacing "*" by "{0,30}" fixed my problem! The regex works for small
    > tables without crashing spamd childs.


    Good to hear it.

    > IMO, upgrading to Perl 5.10 on OSX is not recommended... Apple is
    > tweeking a lot of stuff and there is always chances that a future OSX
    > update will screw up everything.


    Yes -- in general, it can be hard work to do that, and there
    are still the occasional report of bugs in 5.10; if you can avoid it
    for a few more months, do

    --j.

    > many thanks!
    >
    >
    > Dominic Germain
    > ---------------------------------------------
    > Administrateur réseau / Network administrator
    > Sogetel
    > www.sogetel.net
    >
    > mailinglists@sogetel.com
    >
    >
    >
    > Le 08-08-05 à 15:23, Justin Mason a écrit :
    >
    > >
    > > Dominic Germain writes:
    > >> Hi,
    > >>
    > >> I'm running the following rules in my local rules for a while and it
    > >> works flawlessly under RedHat EL 3 (Perl 5.8.0):
    > >>
    > >> rawbody TABLEOBFU /
    > >> ]|"[^"]*"|'[^']*')*>(<([^>]|"[^"]*"|'[^']*')*>)*[a-z]{1,2}
    > >> (<([^>]|"[^"]*"|'[^']*')*>)*<\/td([^>]|"[^"]*"|'[^']*')*>/i
    > >>
    > >> Recently, we move to Apple Mac OS X 10.5 server with Perl 5.8.8. We
    > >> start getting SIGCHLD error and 0/0 scores. I've found that
    > >> TABLEOBFU
    > >> rule crash if there is a large table (~32k) inside the mail... I was
    > >> able to reproduce it under Redhat EL 5 also (Perl 5.8.8) but HTML
    > >> must
    > >> be 2 times larger (~64k).
    > >>
    > >> I also try with the following rule I got from the mailing list. Same
    > >> problem.
    > >>
    > >> rawbody TABLEOBFU /
    > >> ]|"[^"]*"|'[^']*')*>(<([^>]|"[^"]*"|'[^']*')*>)*[a-z]{1,2}
    > >> (<([^>]|"[^"]*"|'[^']*')*>)*<\/td([^>]|"[^"]*"|'[^']*')*>/i
    > >>
    > >> It seems to be something related to allocated memory for the spamd
    > >> script...
    > >>
    > >> Anything known about that issue? Is is possible to "skip" a test
    > >> according to mail size?

    > >
    > > The best fix would be to rewrite those rules not to use /*/, to use
    > > /{0,n}/ instead, and to use /(?...)/ instead of /(...)/ . You
    > > could also
    > > upgrade to perl 5.10, which may have this bug fixed iirc; however you
    > > would still be vulnerable to another side-effect of that kind of
    > > regexp,
    > > which is exponential runtime. unlimited quantifiers like + and * are
    > > very
    > > bad news in SA rules.
    > >
    > > --j.



+ Reply to Thread