Dave Burris wrote:
> OSR5.0.7 MP1 & OSR5.0.5
> Windows XP Pro SP2
> Outlook 2002, Outlook 2003 MP2, and Outlook Express
>
> Two systems in different locations have recently started (last 2 months)
> having the same issue, although to varying degrees. The mail files
> in /usr/spool/mail are getting corrupt and becoming huge. Some mail files
> are becoming corrupt and then grow every time the email client attempts to
> pop email from the server. The email header gets duplicated hundreds of
> times in the spool file and we sometimes see hundreds of lines of
> control-A's (^A^A^A^A). Within a few hours, one or more email files have
> grown to 1GB or 2GB.
>
> On the OSR5.0.7 system, we replaced the SCO popper daemon with FacetWin's
> fct_pop3d daemon. This fixed the email corruption problem and stopped the
> email files from growing, but some users were still getting duplicate messages
> (up to 10 or so sometimes). We then tried using the popper daemon from MP5,
> but immediately started getting corrupt and growing email files again. Went
> back to the fct_pop3d daemon and upgraded the Outlook 2002 clients to Outlook
> 2003 SP2. This solved both problems. Now, we are not getting duplicates or
> corrupted email files.
>
> On the 5.0.5 system, we had the same corruption and growing problem. We
> replaced the popper daemon as discussed here from security fix ERG712033:
> http://groups.google.com/group/comp....4a6eb3121a9e61
> They were not having the problem very often, so it is hard to say if this
> worked, but no problems for a 4 days.
>
> Both of these locations had been running for a couple of years with no
> problems. No apparent virus infections or network issues. We have been
> trying to determine what might have changed recently at either location to
> cause this issue but have not come up with anything.
>
> Has anyone else had this issue lately? Any ideas what causes the corruption
> and file growth?

We too have clients that have been slammed by this in the last 2 weeks.
No solution that I know of. This was the catalyst to move them to a
different method of getting mail...

Fabio

> We too have clients that have been slammed by this in the last 2 weeks.
> No solution that I know of. This was the catalyst to move them to a
> different method of getting mail...
>
> Fabio

For some reason we are see an increase of this as well in the last
couple of weeks. No solution known. All you can do is trash their
mailbox loosing all their mail. I guess your solution, move to a
different mail server, is the only thing that can be done.

I have done a lot of google seaching and searching of the archives.
There are several referances to this problem but no solutions.

Dave Armbrust

Darmbrust wrote:
> > We too have clients that have been slammed by this in the last 2 weeks.
> > No solution that I know of. This was the catalyst to move them to a
> > different method of getting mail...
> >
> > Fabio
>
> For some reason we are see an increase of this as well in the last
> couple of weeks. No solution known. All you can do is trash their
> mailbox loosing all their mail. I guess your solution, move to a
> different mail server, is the only thing that can be done.
>
> I have done a lot of google seaching and searching of the archives.
> There are several referances to this problem but no solutions.
>
> Dave Armbrust

Verify clients are NOT popping from multiple workstations. I have also
seen this when using a pop client and character based mail reader(at
the same time).

Also seem to recall something about lock files. Are their lock files
in the mail directory and what are the permissions on those.

btb

In article <1157135117.996670.174550@m79g2000cwm.googlegroups. com>,
Darmbrust wrote:
>
>> We too have clients that have been slammed by this in the last 2 weeks.
>> No solution that I know of. This was the catalyst to move them to a
>> different method of getting mail...
>>
>> Fabio

>For some reason we are see an increase of this as well in the last
>couple of weeks. No solution known. All you can do is trash their
>mailbox loosing all their mail. I guess your solution, move to a
>different mail server, is the only thing that can be done.

>I have done a lot of google seaching and searching of the archives.
>There are several referances to this problem but no solutions.

I think trashing a users mailbox to lose all the mail is not the
way to make yourself popular.

I've found that such MUA's as 'mutt' will let you go in an
manipulate user mailboxes that get stuck when trying to download
via a popper.

My biggest problem on the servers I run [Not SCO based for email]
are mails with Big5 encoding - and the popper stops on those quite
often. So off into 'mutt', and delete the offending messages, and
let the user gather their real messages.

Considering the content for many of these users deleting and entire
mailbox would cause them to find another service almost
immediately.

And if you think the problem is related only to MMDF, then why not
switch to Sendmail. It's strong and reliable - and a lot of the
bad things you hear about it are stories from long ago.


Bill

--
Bill Vermillion - bv @ wjv . com

> >I have done a lot of google seaching and searching of the archives.
> >There are several referances to this problem but no solutions.
>
> I think trashing a users mailbox to lose all the mail is not the
> way to make yourself popular.

Agreed, but no other solution seems to work. By the time the problem
is brought to my attention the box is huge and corrupt.

> I've found that such MUA's as 'mutt' will let you go in an
> manipulate user mailboxes that get stuck when trying to download
> via a popper.

I am not familar with mutt, but I doublt that any MUA would be able to
handle 28,000 records and mail boxes that are over 2 GB in size! I can
not imagine trying to sort out this number of pieces of mail. If you
think mutt may be of some help please advise where I can find mutt.

> Considering the content for many of these users deleting and entire
> mailbox would cause them to find another service almost
> immediately.

That is EXACTLY what the client is considering and I am really starting
to wonder if using SCO for mail is a mistake.

> And if you think the problem is related only to MMDF, then why not
> switch to Sendmail. It's strong and reliable - and a lot of the
> bad things you hear about it are stories from long ago.

I am already using Sendmail rather then MMDF.

Dave

Darmbrust wrote:
[SNIP]
> I am not familar with mutt, but I doublt that any MUA would be able to
> handle 28,000 records and mail boxes that are over 2 GB in size! I can

^^^^^^^^^^^^There's the problem. Somewhere in comp.mail.* I recall a
discussion about various POP servers breaking over the 2GB limit. You
may want to post there. I don't recall the result, as I use an IMAP
which has 64-bit file support.

Cheers,
Gary B-)

--
__________________________________________________ ____________________________
Armful of chairs: Something some people would not know
whether you were up them with or not
- Barry Humphries

On Tue, Sep 05, 2006, Darmbrust wrote:
>> >I have done a lot of google seaching and searching of the archives.
>> >There are several referances to this problem but no solutions.
>>
>> I think trashing a users mailbox to lose all the mail is not the
>> way to make yourself popular.
>
>Agreed, but no other solution seems to work. By the time the problem
>is brought to my attention the box is huge and corrupt.
>
>> I've found that such MUA's as 'mutt' will let you go in an
>> manipulate user mailboxes that get stuck when trying to download
>> via a popper.
>
>I am not familar with mutt, but I doublt that any MUA would be able to
>handle 28,000 records and mail boxes that are over 2 GB in size! I can
>not imagine trying to sort out this number of pieces of mail. If you
>think mutt may be of some help please advise where I can find mutt.

Single file mail stores aren't really very good when dealing with large
quantities of e-mail messages. Just think of what's required to delete a
single message from the file -- it requires rewriting the entire file.
There are also issues with file locking that are exacerbated when the files
are very large. This is true whether the files are the standard BSD Unix
files, MMDF, U.W. IMAP mbx files, etc.

We have moved all our mail servers to use postfix, amavisd-new, clamav,
courier-authlib, and courier-imap which uses qmail style Maildir for mail
storage. Maildir puts each message in a separate file which simplifies
many things, and it allows standard *nix tools to do things like purge old
messages that are over 90 days old with a command like:

find ~username/Maildir -type f -mtime +90 | xargs rm

We use this with systems ranging from small to medium business to ISPs with
thousands of e-mail clients. My personal Maildir uses nine primary mail
folders, some of which have over 10,000 messages in a folder.

>> Considering the content for many of these users deleting and entire
>> mailbox would cause them to find another service almost
>> immediately.
>
>That is EXACTLY what the client is considering and I am really starting
>to wonder if using SCO for mail is a mistake.

Using SCO for e-mail isn't necessarily a mistake. Using SCO's
standard mail programs is.

We haven't used SCO for much of anything for the last 9 years other than
accounting applications where the vendor only supports SCO or perhaps the
Microsoft Virus, Windows.

The vast majority of our server machines are running SuSE Linux in version
or another with a few FreeBSD boxes thrown in for variety (the machine we
use for our in-house mail server is FreeBSD 4.8 with an uptime of 804 days
now). While I can get most open source software to work with SCO
OpenServer, it requires a lot more effort than doing it on other *nix
platforms -- and I've been building open source software on SCO platforms
for over 20 years now.

>> And if you think the problem is related only to MMDF, then why not
>> switch to Sendmail. It's strong and reliable - and a lot of the
>> bad things you hear about it are stories from long ago.
>
>I am already using Sendmail rather then MMDF.

Sendmail on SCO still uses the MMDF mail stores with ctrl-a message
separators instead of standard BSD-style mailboxes.

Bill
--
INTERNET: bill@Celestial.COM Bill Campbell; Celestial Software LLC
URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way
FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676
http://www.celestial.com/

The question is, why are politicians so eager to be president? What is
it about the job that makes it worth revealing, on national television,
that you have the ethical standards of a slime-coated piece of
industrial waste?
-- Dave Barry, "On Presidential Politics"

What I have been able to find is the version of QPOPPER in SCO 5.0.6 is
causing the problem. By upgrading to an later version of QPOPPER (i.e.
3.1.2) I was able to solve the problem with the growing mailbox. This
does require that you download it from www.qpopper.com and complile it
on your 5.0.6 box. This will at least stop the mail box from growing
to the 2GB system limit.

What seems to kick this off in my system is an large mail box. MS
Outlook is only willing to wait less then a minute for the popper
program after providing an user name and password. It then terminates
the telnet connection which causes the popper program to copy back the
mailbox file which then grows in size due to some sort of bug in the
popper provided in SCO 5.0.6. Every time that Outlook tries to pick
up the mail, and then aborts without waiting for popper to finish
responding, the mailbox increases in size until you hit the system file
size limit. Increasing the Server timeout values in MS Outlook does
not work, it still waits less then aminute.

Again upgrading the popper version to qpopper 3.1.2 fixes the growing
mailbox problem and I assume it works fine in later versions of qpopper
as well but I have not tested these yet.

Not sure what version of qpopper you have? Just telnet to port 110
and it will tell you. With 5.0.6 you will see the line:
+OK QPOP modified by SCO (version 3.0.2-scoR5)

after upgrading to 3.1.2 it shows.

+OK QPOP (version 3.1.2)

I am not sure what versions of popper is used in 5.0.7 and latter
versions so I do not know if this fix applies to them but I would bet
it applied to all 5.0.6 or earlier SCO Unix boxes.

The mailbox file does still seem to get corrupt but at least the
mailbox will no longer grow until you hit the system limit or fill the
file system. The corruption seems to center on the X-UIDL line in the
mailbox. It appears that you can safely remove this line using the
following shell script to fix the corrupt mailbox. Just call the shell
script with the user name as an arguement. I named the shell script
fixmailbox on my system.

/etc/rc2.d/P86sendmail stop
copy -mo /usr/spool/mail/$1 /tmp/$1
sed -e "/X-UIDL: /d" /tmp/$1 >/usr/spool/mail/$1
/etc/rc2.d/P86sendmail stop

It shutsdown sendmail so that you do no get any incomming messages
while the mailbox is being fixed and then restarts it after it is done.
You need to make sure you do not attempt to pick up mail while this
shell script is running. I leaves an unmodified copy of the mailbox in
the /tmp directory which you may want to remove.

Versions of qpopper later then 3.1.2 may fix the corrupt mailbox
problem as well but I am not sure if it is popper that is corrupting
the mailbox. It may be that sendmail is receiving the incomming mail
with a corrupt X-UIDL line or that sendmail itself may be corrupting
the mailbox.

Searching archives this huge/growing/corrupt mailbox problem goes back
to at least 1998 with no solution every presented until now.

In article <1157665593.236843.300820@h48g2000cwc.googlegroups. com>,
Darmbrust wrote:
>I am not sure what versions of popper is used in 5.0.7 and latter
>versions so I do not know if this fix applies to them but I would bet
>it applied to all 5.0.6 or earlier SCO Unix boxes.
>
>The mailbox file does still seem to get corrupt but at least the
>mailbox will no longer grow until you hit the system limit or fill the
>file system. The corruption seems to center on the X-UIDL line in the
>mailbox.

For those running 5.0.7 - a bug involving X-UIDL that resulted in mailbox
corruption was fixed in MP5 (and OSR6 MP2).

John
--
John DuBois spcecdt@armory.com KC6QKZ/AE http://www.armory.com/~spcecdt/