problem with MSGID_MULTIPLE_AT - SpamAssassin

This is a discussion on problem with MSGID_MULTIPLE_AT - SpamAssassin ; Hello. I have a recurrent problem. Many *true* mail are tagged as SPAM because of a too high score. Indeed, a parameter causes problem: MSGID_MULTIPLE_AT is often high I see http://wiki.apache.org/spamassassin/...ID_MULTIPLE_AT But these mails are sent with an Outlook 12.0, ...

+ Reply to Thread
Results 1 to 16 of 16

Thread: problem with MSGID_MULTIPLE_AT

  1. problem with MSGID_MULTIPLE_AT

    Hello.

    I have a recurrent problem. Many *true* mail are tagged as SPAM because of a too high score.
    Indeed, a parameter causes problem:
    MSGID_MULTIPLE_AT is often high

    I see http://wiki.apache.org/spamassassin/...ID_MULTIPLE_AT

    But these mails are sent with an Outlook 12.0, and aren't spam.

    Do you have a solution to solve this problem?

    Thanks for your help,

    Regards,

    --
    -Nicolas.


  2. Re: problem with MSGID_MULTIPLE_AT

    On Tue, 02 Sep 2008 12:51:58 +0200
    Yet Another Ninja wrote:

    > Can you post a sample message on some web server (pastebin.com) so ppl
    > can see what's causing this?
    > PLEASE do NOT munge servernames & IPs


    See the headers:
    http://pastebin.ca/1191372

    I don't have the full message, just headers.
    (I remplaced mails, servernames, etc... by domain.com)

    Do you have a solution? A patch?
    Is it better to disable MSGID_MULTIPLE_AT check?

    Thanks.

    --
    -Nicolas.


  3. Re: problem with MSGID_MULTIPLE_AT

    Yet Another Ninja wrote:
    > On 9/2/2008 1:00 PM, Nicolas Letellier wrote:
    >> On Tue, 02 Sep 2008 12:51:58 +0200
    >> Yet Another Ninja wrote:
    >>
    >>> Can you post a sample message on some web server (pastebin.com) so
    >>> ppl can see what's causing this?
    >>> PLEASE do NOT munge servernames & IPs

    >>
    >> See the headers:
    >> http://pastebin.ca/1191372
    >>
    >> I don't have the full message, just headers.
    >> (I remplaced mails, servernames, etc... by domain.com)

    >
    > unless I'm totally blind and clueless, we're missing a Rcvd header in
    > there.
    > afaik Postfix doesn't do this so what are we missing in that message
    > path?
    > Who is conecting to Postfix?

    Interesting observation, and probably important at some point, but I'd
    treat that as a side note. It's not relevant to the problem at hand.
    >
    >> Do you have a solution? A patch?
    >> Is it better to disable MSGID_MULTIPLE_AT check?

    >
    > I'd would lower the score on that rule till you have it figured out.
    > A "patch" without a Bugzilla entry won't trigger, and it would hardly
    > be an instant fix either.

    Well, it's obvious what the problem is. There's clearly two @ signs in
    the message-id, which is illegal, but it's what Microsoft is doing anyway.

    There's also a bug already open on this.

    https://issues.apache.org/SpamAssass...ug.cgi?id=5707

    We might need to convert that rule to a meta and ignore it when the MUA
    is outlook 12.0 unless we can figure out that the outlook in question
    has some weird hack that causes it, and normal outlook 12 doesn't cause
    the problem.. Although I personally feel makers (and knowing users) of
    broken tools should suffer, I don't think SpamAssassin is the best spot
    to implement that. :-)


  4. Re: problem with MSGID_MULTIPLE_AT

    On Tue, 02 Sep 2008 08:47:18 -0400
    Matt Kettler wrote:
    > Well, it's obvious what the problem is. There's clearly two @ signs in
    > the message-id, which is illegal, but it's what Microsoft is doing anyway.
    >
    > There's also a bug already open on this.
    >
    > https://issues.apache.org/SpamAssass...ug.cgi?id=5707
    >
    > We might need to convert that rule to a meta and ignore it when the MUA
    > is outlook 12.0 unless we can figure out that the outlook in question
    > has some weird hack that causes it, and normal outlook 12 doesn't cause
    > the problem.. Although I personally feel makers (and knowing users) of
    > broken tools should suffer, I don't think SpamAssassin is the best spot
    > to implement that. :-)


    Hello Matt,

    But today, I can't leave this option actived (or not patched). It's important for my business, and too many clients use Outlook 12.0 (I can't force them to use another mail client).

    For waiting, is it possible to disable the MSGID_MULTIPLE_AT check? And how?

    Thanks.


    --
    -Nicolas.


  5. Re: problem with MSGID_MULTIPLE_AT


    Nicolas Letellier writes:
    > On Tue, 02 Sep 2008 08:47:18 -0400
    > Matt Kettler wrote:
    > > Well, it's obvious what the problem is. There's clearly two @ signs in
    > > the message-id, which is illegal, but it's what Microsoft is doing anyway.
    > >
    > > There's also a bug already open on this.
    > >
    > > https://issues.apache.org/SpamAssass...ug.cgi?id=5707
    > >
    > > We might need to convert that rule to a meta and ignore it when the MUA
    > > is outlook 12.0 unless we can figure out that the outlook in question
    > > has some weird hack that causes it, and normal outlook 12 doesn't cause
    > > the problem.. Although I personally feel makers (and knowing users) of
    > > broken tools should suffer, I don't think SpamAssassin is the best spot
    > > to implement that. :-)

    >
    > Hello Matt,
    >
    > But today, I can't leave this option actived (or not patched). It's important for my business, and too many clients use Outlook 12.0 (I can't force them to use another mail client).
    >
    > For waiting, is it possible to disable the MSGID_MULTIPLE_AT check? And how?


    To get it fixed quickly, the most important thing you could do is attach
    multiple samples of messages from Outlook 12.0 which demo this behaviour,
    with full headers and body, to that bug. Right now it appears we have
    none, so no changes to rules can be developed.

    --j.


  6. Re: problem with MSGID_MULTIPLE_AT

    > On Tue, 02 Sep 2008 08:47:18 -0400
    > Matt Kettler wrote:
    >> Well, it's obvious what the problem is. There's clearly two @ signs in
    >> the message-id, which is illegal, but it's what Microsoft is doing anyway.
    >>
    >> There's also a bug already open on this.
    >>
    >> https://issues.apache.org/SpamAssass...ug.cgi?id=5707
    >>
    >> We might need to convert that rule to a meta and ignore it when the MUA
    >> is outlook 12.0 unless we can figure out that the outlook in question
    >> has some weird hack that causes it, and normal outlook 12 doesn't cause
    >> the problem.. Although I personally feel makers (and knowing users) of
    >> broken tools should suffer, I don't think SpamAssassin is the best spot
    >> to implement that. :-)

    >
    > Hello Matt,
    >
    > But today, I can't leave this option actived (or not patched). It's important
    > for my business, and too many clients use Outlook 12.0 (I can't force them to
    > use another mail client).
    >
    > For waiting, is it possible to disable the MSGID_MULTIPLE_AT check? And how?
    >
    > Thanks.
    >

    Add this to local.cf and restart spamd
    score MSGID_MULTIPLE_AT 0

    --
    Michael Scheidell, CTO
    >|SECNAP Network Security

    Winner 2008 Network Products Guide Hot Companies
    FreeBSD SpamAssassin Ports maintainer


    __________________________________________________ _______________________
    This email has been scanned and certified safe by SpammerTrap(r).
    For Information please see http://www.spammertrap.com
    __________________________________________________ _______________________


  7. Re: problem with MSGID_MULTIPLE_AT

    On Tue, 02 Sep 2008 14:12:56 +0100
    jm@jmason.org (Justin Mason) wrote:

    >
    > Nicolas Letellier writes:
    > > On Tue, 02 Sep 2008 08:47:18 -0400
    > > Matt Kettler wrote:
    > > > Well, it's obvious what the problem is. There's clearly two @ signs in
    > > > the message-id, which is illegal, but it's what Microsoft is doing anyway.
    > > >
    > > > There's also a bug already open on this.
    > > >
    > > > https://issues.apache.org/SpamAssass...ug.cgi?id=5707
    > > >
    > > > We might need to convert that rule to a meta and ignore it when the MUA
    > > > is outlook 12.0 unless we can figure out that the outlook in question
    > > > has some weird hack that causes it, and normal outlook 12 doesn't cause
    > > > the problem.. Although I personally feel makers (and knowing users) of
    > > > broken tools should suffer, I don't think SpamAssassin is the best spot
    > > > to implement that. :-)

    > >
    > > Hello Matt,
    > >
    > > But today, I can't leave this option actived (or not patched). It's important for my business, and too many clients use Outlook 12.0 (I can't force them to use another mail client).
    > >
    > > For waiting, is it possible to disable the MSGID_MULTIPLE_AT check? And how?

    >
    > To get it fixed quickly, the most important thing you could do is attach
    > multiple samples of messages from Outlook 12.0 which demo this behaviour,
    > with full headers and body, to that bug. Right now it appears we have
    > none, so no changes to rules can be developed.
    >
    > --j.

    No, I think put a "#" in a file is more quickly than give you full mails and wait for a patch.
    Could you explain me how disable this check? It's important to disable MSGID_MULTIPLE_AT.

    I can send you mails for debug this problem if you want. But not now, because I must ask them to my users.

    Thanks !


    --
    -Nicolas.


  8. Re: problem with MSGID_MULTIPLE_AT

    Hi,

    Michael Scheidell wrote:
    >> On Tue, 02 Sep 2008 08:47:18 -0400
    >> Matt Kettler wrote:
    >>> Well, it's obvious what the problem is. There's clearly two @ signs in
    >>> the message-id, which is illegal, but it's what Microsoft is doing anyway.
    >>>
    >>> There's also a bug already open on this.
    >>>
    >>> https://issues.apache.org/SpamAssass...ug.cgi?id=5707
    >>>
    >>> We might need to convert that rule to a meta and ignore it when the MUA
    >>> is outlook 12.0 unless we can figure out that the outlook in question
    >>> has some weird hack that causes it, and normal outlook 12 doesn't cause
    >>> the problem.. Although I personally feel makers (and knowing users) of
    >>> broken tools should suffer, I don't think SpamAssassin is the best spot
    >>> to implement that. :-)

    >> Hello Matt,
    >>
    >> But today, I can't leave this option actived (or not patched). It's important
    >> for my business, and too many clients use Outlook 12.0 (I can't force them to
    >> use another mail client).
    >>
    >> For waiting, is it possible to disable the MSGID_MULTIPLE_AT check? And how?
    >>
    >> Thanks.
    >>

    > Add this to local.cf and restart spamd
    > score MSGID_MULTIPLE_AT 0


    Or give it a very small positive score, so you can at least see when it
    is firing, but it won't have a large impact on the overall score.

    score MSGID_MULTIPLE_AT 0.1

    --
    Anthony Pea****
    CHIME, Royal Free & University College Medical School
    WWW: http://www.chime.ucl.ac.uk/~rmhiajp/
    Study Health Informatics - Modular Postgraduate Degree
    http://www.chime.ucl.ac.uk/study-health-informatics/


  9. Re: problem with MSGID_MULTIPLE_AT

    On Tue, 2008-09-02 at 13:00 +0200, Nicolas Letellier wrote:
    > On Tue, 02 Sep 2008 12:51:58 +0200 Yet Another Ninja wrote:
    >
    > > Can you post a sample message on some web server (pastebin.com) so ppl
    > > can see what's causing this?
    > > PLEASE do NOT munge servernames & IPs

    >
    > See the headers:
    > http://pastebin.ca/1191372
    >
    > I don't have the full message, just headers.
    > (I remplaced mails, servernames, etc... by domain.com)


    It appears you got greater problems than that rule. Have a closer look
    at the Report.

    One problem is Bayes, which probably could be trained better. That
    message scored BAYES_50. If you frequently discuss similar topics by
    mail, Bayes should considerably lean towards 00.

    The most glaring problem is AWL, though. Without AWL, that message
    already scored 3.1, which is pretty high for a ham, but still no
    problem. Yes, that includes the rule in question.

    Now, AWL accounts for another 2.9 points. This means that you previously
    got mail by that sender, and it appeared to score much higher. The way
    AWL works -- if you received, say, more than 2 messages by that user
    before -- setting MSGID_MULTIPLE_AT to 0 will *not* help, since AWL is
    sure to skyrocket the score above your threshold of 5 again.


    Things to consider: (a) Train your ham, in particular FPs like this and
    any other important mail. (b) Drop that user from your AWL database,
    for an immediate fix. And last but not least (c) figure out why the
    *average* score of all mail sent by that users scores way *above* 6.

    The last part is most important. AWL is a score averager. So previous
    messages by that sender scored even higher than the sample you provided.
    That's where you need to look into.


    > Do you have a solution? A patch?
    > Is it better to disable MSGID_MULTIPLE_AT check?


    score MSGID_MULTIPLE_AT 0

    guenther


    --
    char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a \x10\xf4\xf4\xc4";
    main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


  10. Re: problem with MSGID_MULTIPLE_AT

    On Tue, 2008-09-02 at 15:23 +0200, Nicolas Letellier wrote:

    > No, I think put a "#" in a file is more quickly than give you full
    > mails and wait for a patch.
    > Could you explain me how disable this check? It's important to disable
    > MSGID_MULTIPLE_AT.


    Just as has been mentioned by others already, set the rule's score to 0.
    Do NOT comment out the rule in the stock (update) cf files. That will
    break on your next sa-update run.

    Anyway, I believe just disabling this rule won't help much. See my other
    post with details about this.

    guenther


    --
    char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a \x10\xf4\xf4\xc4";
    main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


  11. Re: problem with MSGID_MULTIPLE_AT

    On Tue, 02 Sep 2008 15:40:18 +0200
    Karsten Brńckelmann wrote:

    > On Tue, 2008-09-02 at 13:00 +0200, Nicolas Letellier wrote:
    > > On Tue, 02 Sep 2008 12:51:58 +0200 Yet Another Ninja wrote:
    > >
    > > > Can you post a sample message on some web server (pastebin.com) so ppl
    > > > can see what's causing this?
    > > > PLEASE do NOT munge servernames & IPs

    > >
    > > See the headers:
    > > http://pastebin.ca/1191372
    > >
    > > I don't have the full message, just headers.
    > > (I remplaced mails, servernames, etc... by domain.com)

    >
    > It appears you got greater problems than that rule. Have a closer look
    > at the Report.
    >
    > One problem is Bayes, which probably could be trained better. That
    > message scored BAYES_50. If you frequently discuss similar topics by
    > mail, Bayes should considerably lean towards 00.
    >
    > The most glaring problem is AWL, though. Without AWL, that message
    > already scored 3.1, which is pretty high for a ham, but still no
    > problem. Yes, that includes the rule in question.
    >
    > Now, AWL accounts for another 2.9 points. This means that you previously
    > got mail by that sender, and it appeared to score much higher. The way
    > AWL works -- if you received, say, more than 2 messages by that user
    > before -- setting MSGID_MULTIPLE_AT to 0 will *not* help, since AWL is
    > sure to skyrocket the score above your threshold of 5 again.
    >
    >
    > Things to consider: (a) Train your ham, in particular FPs like this and
    > any other important mail. (b) Drop that user from your AWL database,
    > for an immediate fix. And last but not least (c) figure out why the
    > *average* score of all mail sent by that users scores way *above* 6.
    >
    > The last part is most important. AWL is a score averager. So previous
    > messages by that sender scored even higher than the sample you provided.
    > That's where you need to look into.



    Hi Karsten,

    Thanks for your complete message.

    I dropped this user from AWL database (with "spamassassin --remove-addr-from-whitelist").

    On the other hand, the mail I've pasted was just an example. I have many mail not tagged as spam, but with a big MSGID_MULTIPLE_AT. It could be a problem.

    Regards,


    --
    -Nicolas.


  12. Re: problem with MSGID_MULTIPLE_AT

    On Tue, 02 Sep 2008 15:45:35 +0200
    Karsten Brńckelmann wrote:

    > On Tue, 2008-09-02 at 15:23 +0200, Nicolas Letellier wrote:
    >
    > > No, I think put a "#" in a file is more quickly than give you full
    > > mails and wait for a patch.
    > > Could you explain me how disable this check? It's important to disable
    > > MSGID_MULTIPLE_AT.

    >
    > Just as has been mentioned by others already, set the rule's score to 0.
    > Do NOT comment out the rule in the stock (update) cf files. That will
    > break on your next sa-update run.
    >
    > Anyway, I believe just disabling this rule won't help much. See my other
    > post with details about this.


    Thanks for the line (and others who told it too).
    Indeed, this line will not help me for the mail I've pasted. Just a little.But it's better than now, waiting a patch.

    Regards,

    --
    -Nicolas.


  13. Re: problem with MSGID_MULTIPLE_AT

    On Tue, 2008-09-02 at 15:53 +0200, Nicolas Letellier wrote:
    > On Tue, 02 Sep 2008 15:40:18 +0200 Karsten Br├Ąckelmann wrote:


    > > Things to consider: (a) Train your ham, in particular FPs like this and
    > > any other important mail. (b) Drop that user from your AWL database,
    > > for an immediate fix. And last but not least (c) figure out why the
    > > *average* score of all mail sent by that users scores way *above* 6.
    > >
    > > The last part is most important. AWL is a score averager. So previous
    > > messages by that sender scored even higher than the sample you provided.
    > > That's where you need to look into.

    >
    > Hi Karsten,
    >
    > Thanks for your complete message.
    >
    > I dropped this user from AWL database (with "spamassassin --remove-addr-from-whitelist").
    >
    > On the other hand, the mail I've pasted was just an example. I have
    > many mail not tagged as spam, but with a big MSGID_MULTIPLE_AT. It
    > could be a problem.


    Exactly my point. While that rule indeed is a heavy weight in your
    scores, it is not necessarily the root cause. AWL sticks out like a sore
    thumb in the example you posted.

    At the very least, it is a combination of that rule, a seriously bad AWL
    history [1] and sub-optimal Bayes training.

    guenther


    [1] You *did* get FPs from that sender in the past with scores above 6.
    If you didn't spot them before, look out for them in your spam
    folder.

    --
    char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a \x10\xf4\xf4\xc4";
    main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


  14. Re: problem with MSGID_MULTIPLE_AT

    On Tue, 2008-09-02 at 16:02 +0200, Nicolas Letellier wrote:
    > On Tue, 02 Sep 2008 15:45:35 +0200 Karsten Br├Ąckelmann wrote:


    > > Anyway, I believe just disabling this rule won't help much. See my other
    > > post with details about this.

    >
    > Thanks for the line (and others who told it too).
    > Indeed, this line will not help me for the mail I've pasted. Just a
    > little. But it's better than now, waiting a patch.


    Other than proper Bayes ´╗┐training, which has been mentioned already, and
    investigating what caused previous messages to score that high...

    If you frequently get high scoring mail from some senders for whatever
    weird reason, it might be worth looking into whitelist_from_rcvd.

    guenther


    --
    char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a \x10\xf4\xf4\xc4";
    main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


  15. Re: problem with MSGID_MULTIPLE_AT

    Nicolas Letellier wrote:
    > On Tue, 02 Sep 2008 08:47:18 -0400
    > Matt Kettler wrote:
    >
    >> Well, it's obvious what the problem is. There's clearly two @ signs in
    >> the message-id, which is illegal, but it's what Microsoft is doing anyway.
    >>
    >> There's also a bug already open on this.
    >>
    >> https://issues.apache.org/SpamAssass...ug.cgi?id=5707
    >>
    >> We might need to convert that rule to a meta and ignore it when the MUA
    >> is outlook 12.0 unless we can figure out that the outlook in question
    >> has some weird hack that causes it, and normal outlook 12 doesn't cause
    >> the problem.. Although I personally feel makers (and knowing users) of
    >> broken tools should suffer, I don't think SpamAssassin is the best spot
    >> to implement that. :-)
    >>

    >
    > Hello Matt,
    >
    > But today, I can't leave this option actived (or not patched). It's important for my business, and too many clients use Outlook 12.0 (I can't force them to use another mail client).
    >
    > For waiting, is it possible to disable the MSGID_MULTIPLE_AT check? And how?
    >
    > Thanks.
    >
    >
    >

    The best way to disable a rule, is to add a score statement for it
    setting it's score to 0 in your local.cf

    score MSGID_MULTIPLE_AT 0

    and yes, that does disable it. SA won't even evaluate a rule explicitly
    set to zero sc0re


  16. Re: problem with MSGID_MULTIPLE_AT


    I looked for any cases of MSGID_MULTIPLE_AT logged as spam yesterday.
    We log if it scores 7.0 or higher. A message from a legit company
    called Mathworks had this Message-ID, as logged by sendmail:

    msgid=>

    That hit INVALID_MSGID and MSGID_MULTIPLE_AT. My reading of RFC 2822
    is that the < and > and @ are not allowed in the RHS. I don't know
    what software produced this. It was sent to many recipients, so I
    think it was a mass mailing product. I don't have the message itself.

    Joseph Brennan
    Lead Email Systems Engineer
    Columbia University Information Technology



+ Reply to Thread