Hello,

I am running SuSE 8.2 (fully patched) on a Dell PowerEdge 2650 with an
external PowerVault RAID array on an LSI-based PERC2 card. The server
shares files via Samba and NFS, and also hosts several moderately
intensive applications for gene sequencing and mapping. The box is a
dual Xeon with 4GB of RAM, with root and swap on an internal HW mirror
and 400GB of data on the external RAID.

Users have been complaining for some time now that the box hesitates
in some situations, particularly when using "tab" to autocomplete
commands and/or file names, and when entering any command for the
first time in a while. That is, cd'ing to and ls'ing a directory for
the first time might cause the machine to pause for anywhere from 2 to
10 seconds. Oddly, the second time there's no hesitation. Ths pause
occurs when executing commands at the shell, when connecting to files
via samba, and when acccessing server applications via an X-based
client.

Everything I can find for performance stats suggests that the box is
not hitting even a significant portion of its resources. Top, sar and
iostat all show that the CPUs have never been more than 30% occupied,
"free" shows that a gig of physical memory is available, and the swap
partition (2GB) has only used about 40MB of its space.

I've checked to see if any of the problem users have folders in their
path that include a large number of files. None are more than 1200 or
so and most are much less (20-200).

I've confirmed that this occurs when ls'ing local volumes from the
console (suggesting it's not a network problem?).

This problem has persisted across several kernel updates.

I have a feeling this is probably not related to the RAID array or the
amount of data on it as I've seen some signs of the same behavior from
a different Dell (also running 8.2) with no external storage.

"iostat -x -d" did show some heavier stats for the root drive,
especially the swap partition, and the fact that this appears to be
related to caching of commands and filenames made me wonder if
caching/paging/swapping was bogging down somewhere. As a shot in the
dark I added another physical drive, configured it as a single (18GB)
swap partition, and moved all swapping to it. No difference.

I'm aware that this is not enough information to make any real guesses
(I've been googling on this for a week and still haven't found even
any similar problems, let alone a solution). I welcome any suggestions
or feedback, however, and am especially hopeful that someone else may
have seen this and found its cause.

Randall Rue
System Administrator
IT, Server Operations
Fred Hutchinson Cancer Research Center
Seattle, WA USA