> > after booting a redhat enterprise linux 3 machine with apache 2.0.58,
> > perl 5.8.8 and mod_perl 2.0.2,
> > it runs well using about 300 M of 1 G physical RAM.
> > However, the remaining RAM decreases day by day, and after 2 or 3
> > weeks, the machine crashes because swapping takes too much time.
> > However, all processes together take about 250 MBytes according to ps,
> > thus I assume that the kernel takes the rest. free tells me in fact
> > that much swap space is used an nearly no physical RAM is left.
> >
> > The strange thing is that all the memory is gone even after
> > stopping all apache processes.
> > The only thing which helps is to reboot the machine.

>
>
> I'm experiencing nearly the same thing now on
> FreeBSD 6.0 REL
> Apache/2.0.58
> mod_perl 2.0.2
> Postgres 8.1.3


I wish I'd caught this thread from the beginning. I've been seeing
this exact same problem for several months now. Here is the
environment where I first noticed it:

Linux 2.6.12.6
Apache 1.3.33
Postgresql 7.4.9
mod_perl 1.29
350-400 viritual hosted domains, all running a mod_perl/postgres backed CMS.

To make it clear: The memory is definitely lost. After 2-3 weeks on a
machine with 2GB of RAM, I can shut down *every single daemon* on the
machine and the RAM is not freed. And it is not a misreading of "top"
or "free". That memory is gone, and trying to keep the machine running
results in it going heavily into swap just for basic processes, and
eventually crashing.

It looks a lot like a kernel memory leak, and for the past few months
that is what I've assumed it was. I don't know how else to explain the
fact that no processes are running but the kernel still sees all the
RAM taken. However, the fact that it was reported on 2.4 *and* BSD
here makes me reconsider.

Andreas: Are you also running PostgreSQL as Jonathan and I are?

Andreas & Jonathan: Are either of you running reiserfs? (Perhaps it is
the filesystem causing this.)

Ok, now possibly some light at the end of the tunnel: I have set up a
second machine with the following specs:

Linux 2.6.16.27
Postgresql 7.4.13
Apache 1.3.37
mod_perl 1.29
~100 virtual hosted domains, all running mod_perl/postgres CMS

I have not seen any memory issues with this machine. When all of the
processes are stopped, the memory usage goes right back down to where
it should be. However, I don't recall seeing the memory problems on
the other machine until it hit around 300-350 vhosted domains. But
then again, I wasn't paying very much attention until it actually
became a problem. Right now I'd say it's a little too early to tell,
but in a couple months I should have ~300 domains on this box as well,
and then I'll know if this particular configuration is immune to the
memory problems.

Also, I've eliminated Apache as the cause, as the following
configuration also has the memory problem:

Linux 2.6.12.6
Apache 1.3.37
Postgresql 7.4.9
mod_perl 1.29

Perhaps this weekend I'll upgrade the postgres on this box to 7.4.13
and see if it makes a difference. If not, the only thing left is the
kernel...