popper/corrupt email - SCO

This is a discussion on popper/corrupt email - SCO ; OSR5.0.7 MP1 & OSR5.0.5 Windows XP Pro SP2 Outlook 2002, Outlook 2003 MP2, and Outlook Express Two systems in different locations have recently started (last 2 months) having the same issue, although to varying degrees. The mail files in /usr/spool/mail ...

+ Reply to Thread
Results 1 to 10 of 10

Thread: popper/corrupt email

  1. popper/corrupt email

    OSR5.0.7 MP1 & OSR5.0.5
    Windows XP Pro SP2
    Outlook 2002, Outlook 2003 MP2, and Outlook Express

    Two systems in different locations have recently started (last 2 months)
    having the same issue, although to varying degrees. The mail files
    in /usr/spool/mail are getting corrupt and becoming huge. Some mail files
    are becoming corrupt and then grow every time the email client attempts to
    pop email from the server. The email header gets duplicated hundreds of
    times in the spool file and we sometimes see hundreds of lines of
    control-A's (^A^A^A^A). Within a few hours, one or more email files have
    grown to 1GB or 2GB.

    On the OSR5.0.7 system, we replaced the SCO popper daemon with FacetWin's
    fct_pop3d daemon. This fixed the email corruption problem and stopped the
    email files from growing, but some users were still getting duplicate messages
    (up to 10 or so sometimes). We then tried using the popper daemon from MP5,
    but immediately started getting corrupt and growing email files again. Went
    back to the fct_pop3d daemon and upgraded the Outlook 2002 clients to Outlook
    2003 SP2. This solved both problems. Now, we are not getting duplicates or
    corrupted email files.

    On the 5.0.5 system, we had the same corruption and growing problem. We
    replaced the popper daemon as discussed here from security fix ERG712033:
    http://groups.google.com/group/comp....4a6eb3121a9e61
    They were not having the problem very often, so it is hard to say if this
    worked, but no problems for a 4 days.

    Both of these locations had been running for a couple of years with no
    problems. No apparent virus infections or network issues. We have been
    trying to determine what might have changed recently at either location to
    cause this issue but have not come up with anything.

    Has anyone else had this issue lately? Any ideas what causes the corruption
    and file growth?

    Thanks,
    Dave

  2. Re: popper/corrupt email


    Dave Burris wrote:
    > OSR5.0.7 MP1 & OSR5.0.5
    > Windows XP Pro SP2
    > Outlook 2002, Outlook 2003 MP2, and Outlook Express
    >
    > Two systems in different locations have recently started (last 2 months)
    > having the same issue, although to varying degrees. The mail files
    > in /usr/spool/mail are getting corrupt and becoming huge. Some mail files
    > are becoming corrupt and then grow every time the email client attempts to
    > pop email from the server. The email header gets duplicated hundreds of
    > times in the spool file and we sometimes see hundreds of lines of
    > control-A's (^A^A^A^A). Within a few hours, one or more email files have
    > grown to 1GB or 2GB.
    >
    > On the OSR5.0.7 system, we replaced the SCO popper daemon with FacetWin's
    > fct_pop3d daemon. This fixed the email corruption problem and stopped the
    > email files from growing, but some users were still getting duplicate messages
    > (up to 10 or so sometimes). We then tried using the popper daemon from MP5,
    > but immediately started getting corrupt and growing email files again. Went
    > back to the fct_pop3d daemon and upgraded the Outlook 2002 clients to Outlook
    > 2003 SP2. This solved both problems. Now, we are not getting duplicates or
    > corrupted email files.
    >
    > On the 5.0.5 system, we had the same corruption and growing problem. We
    > replaced the popper daemon as discussed here from security fix ERG712033:
    > http://groups.google.com/group/comp....4a6eb3121a9e61
    > They were not having the problem very often, so it is hard to say if this
    > worked, but no problems for a 4 days.
    >
    > Both of these locations had been running for a couple of years with no
    > problems. No apparent virus infections or network issues. We have been
    > trying to determine what might have changed recently at either location to
    > cause this issue but have not come up with anything.
    >
    > Has anyone else had this issue lately? Any ideas what causes the corruption
    > and file growth?


    We too have clients that have been slammed by this in the last 2 weeks.
    No solution that I know of. This was the catalyst to move them to a
    different method of getting mail...

    Fabio


  3. Re: popper/corrupt email


    > We too have clients that have been slammed by this in the last 2 weeks.
    > No solution that I know of. This was the catalyst to move them to a
    > different method of getting mail...
    >
    > Fabio


    For some reason we are see an increase of this as well in the last
    couple of weeks. No solution known. All you can do is trash their
    mailbox loosing all their mail. I guess your solution, move to a
    different mail server, is the only thing that can be done.

    I have done a lot of google seaching and searching of the archives.
    There are several referances to this problem but no solutions.

    Dave Armbrust


  4. Re: popper/corrupt email

    Darmbrust wrote:
    > > We too have clients that have been slammed by this in the last 2 weeks.
    > > No solution that I know of. This was the catalyst to move them to a
    > > different method of getting mail...
    > >
    > > Fabio

    >
    > For some reason we are see an increase of this as well in the last
    > couple of weeks. No solution known. All you can do is trash their
    > mailbox loosing all their mail. I guess your solution, move to a
    > different mail server, is the only thing that can be done.
    >
    > I have done a lot of google seaching and searching of the archives.
    > There are several referances to this problem but no solutions.
    >
    > Dave Armbrust


    Verify clients are NOT popping from multiple workstations. I have also
    seen this when using a pop client and character based mail reader(at
    the same time).

    Also seem to recall something about lock files. Are their lock files
    in the mail directory and what are the permissions on those.

    btb


  5. Re: popper/corrupt email

    In article <1157135117.996670.174550@m79g2000cwm.googlegroups. com>,
    Darmbrust wrote:
    >
    >> We too have clients that have been slammed by this in the last 2 weeks.
    >> No solution that I know of. This was the catalyst to move them to a
    >> different method of getting mail...
    >>
    >> Fabio


    >For some reason we are see an increase of this as well in the last
    >couple of weeks. No solution known. All you can do is trash their
    >mailbox loosing all their mail. I guess your solution, move to a
    >different mail server, is the only thing that can be done.


    >I have done a lot of google seaching and searching of the archives.
    >There are several referances to this problem but no solutions.


    I think trashing a users mailbox to lose all the mail is not the
    way to make yourself popular.

    I've found that such MUA's as 'mutt' will let you go in an
    manipulate user mailboxes that get stuck when trying to download
    via a popper.

    My biggest problem on the servers I run [Not SCO based for email]
    are mails with Big5 encoding - and the popper stops on those quite
    often. So off into 'mutt', and delete the offending messages, and
    let the user gather their real messages.

    Considering the content for many of these users deleting and entire
    mailbox would cause them to find another service almost
    immediately.

    And if you think the problem is related only to MMDF, then why not
    switch to Sendmail. It's strong and reliable - and a lot of the
    bad things you hear about it are stories from long ago.


    Bill

    --
    Bill Vermillion - bv @ wjv . com

  6. Re: popper/corrupt email

    > >I have done a lot of google seaching and searching of the archives.
    > >There are several referances to this problem but no solutions.

    >
    > I think trashing a users mailbox to lose all the mail is not the
    > way to make yourself popular.


    Agreed, but no other solution seems to work. By the time the problem
    is brought to my attention the box is huge and corrupt.

    > I've found that such MUA's as 'mutt' will let you go in an
    > manipulate user mailboxes that get stuck when trying to download
    > via a popper.


    I am not familar with mutt, but I doublt that any MUA would be able to
    handle 28,000 records and mail boxes that are over 2 GB in size! I can
    not imagine trying to sort out this number of pieces of mail. If you
    think mutt may be of some help please advise where I can find mutt.

    > Considering the content for many of these users deleting and entire
    > mailbox would cause them to find another service almost
    > immediately.


    That is EXACTLY what the client is considering and I am really starting
    to wonder if using SCO for mail is a mistake.

    > And if you think the problem is related only to MMDF, then why not
    > switch to Sendmail. It's strong and reliable - and a lot of the
    > bad things you hear about it are stories from long ago.


    I am already using Sendmail rather then MMDF.

    Dave


  7. Re: popper/corrupt email

    Darmbrust wrote:
    [SNIP]
    > I am not familar with mutt, but I doublt that any MUA would be able to
    > handle 28,000 records and mail boxes that are over 2 GB in size! I can


    ^^^^^^^^^^^^There's the problem. Somewhere in comp.mail.* I recall a
    discussion about various POP servers breaking over the 2GB limit. You
    may want to post there. I don't recall the result, as I use an IMAP
    which has 64-bit file support.

    Cheers,
    Gary B-)

    --
    __________________________________________________ ____________________________
    Armful of chairs: Something some people would not know
    whether you were up them with or not
    - Barry Humphries

  8. Re: popper/corrupt email

    On Tue, Sep 05, 2006, Darmbrust wrote:
    >> >I have done a lot of google seaching and searching of the archives.
    >> >There are several referances to this problem but no solutions.

    >>
    >> I think trashing a users mailbox to lose all the mail is not the
    >> way to make yourself popular.

    >
    >Agreed, but no other solution seems to work. By the time the problem
    >is brought to my attention the box is huge and corrupt.
    >
    >> I've found that such MUA's as 'mutt' will let you go in an
    >> manipulate user mailboxes that get stuck when trying to download
    >> via a popper.

    >
    >I am not familar with mutt, but I doublt that any MUA would be able to
    >handle 28,000 records and mail boxes that are over 2 GB in size! I can
    >not imagine trying to sort out this number of pieces of mail. If you
    >think mutt may be of some help please advise where I can find mutt.


    Single file mail stores aren't really very good when dealing with large
    quantities of e-mail messages. Just think of what's required to delete a
    single message from the file -- it requires rewriting the entire file.
    There are also issues with file locking that are exacerbated when the files
    are very large. This is true whether the files are the standard BSD Unix
    files, MMDF, U.W. IMAP mbx files, etc.

    We have moved all our mail servers to use postfix, amavisd-new, clamav,
    courier-authlib, and courier-imap which uses qmail style Maildir for mail
    storage. Maildir puts each message in a separate file which simplifies
    many things, and it allows standard *nix tools to do things like purge old
    messages that are over 90 days old with a command like:

    find ~username/Maildir -type f -mtime +90 | xargs rm

    We use this with systems ranging from small to medium business to ISPs with
    thousands of e-mail clients. My personal Maildir uses nine primary mail
    folders, some of which have over 10,000 messages in a folder.

    >> Considering the content for many of these users deleting and entire
    >> mailbox would cause them to find another service almost
    >> immediately.

    >
    >That is EXACTLY what the client is considering and I am really starting
    >to wonder if using SCO for mail is a mistake.


    Using SCO for e-mail isn't necessarily a mistake. Using SCO's
    standard mail programs is.

    We haven't used SCO for much of anything for the last 9 years other than
    accounting applications where the vendor only supports SCO or perhaps the
    Microsoft Virus, Windows.

    The vast majority of our server machines are running SuSE Linux in version
    or another with a few FreeBSD boxes thrown in for variety (the machine we
    use for our in-house mail server is FreeBSD 4.8 with an uptime of 804 days
    now). While I can get most open source software to work with SCO
    OpenServer, it requires a lot more effort than doing it on other *nix
    platforms -- and I've been building open source software on SCO platforms
    for over 20 years now.

    >> And if you think the problem is related only to MMDF, then why not
    >> switch to Sendmail. It's strong and reliable - and a lot of the
    >> bad things you hear about it are stories from long ago.

    >
    >I am already using Sendmail rather then MMDF.


    Sendmail on SCO still uses the MMDF mail stores with ctrl-a message
    separators instead of standard BSD-style mailboxes.

    Bill
    --
    INTERNET: bill@Celestial.COM Bill Campbell; Celestial Software LLC
    URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way
    FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676
    http://www.celestial.com/

    The question is, why are politicians so eager to be president? What is
    it about the job that makes it worth revealing, on national television,
    that you have the ethical standards of a slime-coated piece of
    industrial waste?
    -- Dave Barry, "On Presidential Politics"

  9. Re: popper/corrupt email - PROBLEM SOLVED

    What I have been able to find is the version of QPOPPER in SCO 5.0.6 is
    causing the problem. By upgrading to an later version of QPOPPER (i.e.
    3.1.2) I was able to solve the problem with the growing mailbox. This
    does require that you download it from www.qpopper.com and complile it
    on your 5.0.6 box. This will at least stop the mail box from growing
    to the 2GB system limit.

    What seems to kick this off in my system is an large mail box. MS
    Outlook is only willing to wait less then a minute for the popper
    program after providing an user name and password. It then terminates
    the telnet connection which causes the popper program to copy back the
    mailbox file which then grows in size due to some sort of bug in the
    popper provided in SCO 5.0.6. Every time that Outlook tries to pick
    up the mail, and then aborts without waiting for popper to finish
    responding, the mailbox increases in size until you hit the system file
    size limit. Increasing the Server timeout values in MS Outlook does
    not work, it still waits less then aminute.

    Again upgrading the popper version to qpopper 3.1.2 fixes the growing
    mailbox problem and I assume it works fine in later versions of qpopper
    as well but I have not tested these yet.

    Not sure what version of qpopper you have? Just telnet to port 110
    and it will tell you. With 5.0.6 you will see the line:
    +OK QPOP modified by SCO (version 3.0.2-scoR5)

    after upgrading to 3.1.2 it shows.

    +OK QPOP (version 3.1.2)

    I am not sure what versions of popper is used in 5.0.7 and latter
    versions so I do not know if this fix applies to them but I would bet
    it applied to all 5.0.6 or earlier SCO Unix boxes.

    The mailbox file does still seem to get corrupt but at least the
    mailbox will no longer grow until you hit the system limit or fill the
    file system. The corruption seems to center on the X-UIDL line in the
    mailbox. It appears that you can safely remove this line using the
    following shell script to fix the corrupt mailbox. Just call the shell
    script with the user name as an arguement. I named the shell script
    fixmailbox on my system.

    /etc/rc2.d/P86sendmail stop
    copy -mo /usr/spool/mail/$1 /tmp/$1
    sed -e "/X-UIDL: /d" /tmp/$1 >/usr/spool/mail/$1
    /etc/rc2.d/P86sendmail stop

    It shutsdown sendmail so that you do no get any incomming messages
    while the mailbox is being fixed and then restarts it after it is done.
    You need to make sure you do not attempt to pick up mail while this
    shell script is running. I leaves an unmodified copy of the mailbox in
    the /tmp directory which you may want to remove.

    Versions of qpopper later then 3.1.2 may fix the corrupt mailbox
    problem as well but I am not sure if it is popper that is corrupting
    the mailbox. It may be that sendmail is receiving the incomming mail
    with a corrupt X-UIDL line or that sendmail itself may be corrupting
    the mailbox.

    Searching archives this huge/growing/corrupt mailbox problem goes back
    to at least 1998 with no solution every presented until now.


  10. Re: popper/corrupt email - PROBLEM SOLVED

    In article <1157665593.236843.300820@h48g2000cwc.googlegroups. com>,
    Darmbrust wrote:
    >I am not sure what versions of popper is used in 5.0.7 and latter
    >versions so I do not know if this fix applies to them but I would bet
    >it applied to all 5.0.6 or earlier SCO Unix boxes.
    >
    >The mailbox file does still seem to get corrupt but at least the
    >mailbox will no longer grow until you hit the system limit or fill the
    >file system. The corruption seems to center on the X-UIDL line in the
    >mailbox.


    For those running 5.0.7 - a bug involving X-UIDL that resulted in mailbox
    corruption was fixed in MP5 (and OSR6 MP2).

    John
    --
    John DuBois spcecdt@armory.com KC6QKZ/AE http://www.armory.com/~spcecdt/

+ Reply to Thread