We've been using samba at $work for as long as I've been here but
recently we've been experiencing some weird load related issues.

The platform is FreeBSD 6.2 with a really basic Samba share to a
simple nobody:nogroup share spanning an entire disk array (between 1-4
TB). We've moved a synching application (a bunch of client software
configuration is sync'd over from a global share) which has every
machine in the office hitting it quite frequently. When on served
from our lone Windows server (W2KAS) it worked albeit slowly at the
worst loads (think 250 to 300 clients at peak). Now moving it over to
a FreeBSD 6.2 machine this single application has brought the server
to it's knees. We get connection timeouts and weird behaviour such as
random disconnects and 'delayed write failures.'

The machine is not lacking for resources CPUs largely idle, lots of
free memory etc.
Originally I thought this was do to maxfiles, maxfileperproc and
somaxcon sysctls but since bumping those values the machine has been
more responsive but the wierd errors persist.

I also cannot consistently replicate it which is making it hard to
determine the state.

Is there anything I can look at to help track down this scaling issue?
As far as I know samba should just core at high connection rates.
The disks are fine as reported by the drive array and the array is
used frequently for other high volume traffic.

The logs show stuff like:
[2007/05/29 14:43:14, 0] lib/util_sock.c:read_data(534) read_data:
read failure for 4 bytes to client 10.0.2.104. Error = Connection
reset by peer[2007/05/29 14:43:17, 0] smbd/
reply.c:send_file_readX(2621) send_file_readX: sendfile failed for
file hal/max9_synch/Main/Win32/synch_plugins/3rdparty_plugins/
Afterworks/FumeFxSL/FumeFX.dlo (Broken pipe). Terminating
[2007/05/29 14:43:17, 0] lib/util_sock.c:read_data(534)
read_data: read failure for 4 bytes to client 10.0.2.78. Error =
Connection reset by peer
[2007/05/29 14:43:17, 0] smbd/server.c:exit_server_common(657)
[2007/05/29 14:43:08, 5] lib/util.c:show_msg(488) size=59
smb_com=0x2e smb_rcls=0 smb_reh=0 smb_err=0 smb_flg=24
smb_flg2=55303 smb_tid=2 smb_pid=65279 smb_uid=0 smb_mid=1985
smt_wct=12 smb_vwv[ 0]= 255 (0xFF) smb_vwv[ 1]=57054 (0xDEDE)
smb_vwv[ 2]=13461 (0x3495) smb_vwv[ 3]= 0 (0x0)
smb_vwv[ 4]= 2 (0x2) smb_vwv[ 5]=61440 (0xF000)
smb_vwv[ 6]=61440 (0xF000) smb_vwv[ 7]= 0 (0x0)
smb_vwv[ 8]= 0 (0x0) smb_vwv[ 9]= 0 (0x0) smb_vwv[10]= 0
(0x0) smb_vwv[11]= 0 (0x0) smb_bcc=0[2007/05/29 14:43:08, 0] lib/
fault.c:dump_core(173) dumping core in /var/log/samba/cores/smbd

smbd -V -> Version 3.0.23c

/etc/sysctl.conf
kern.ipc.maxsockbuf=8388608
net.inet.tcp.sendspace=65535
net.inet.tcp.recvspace=65535
net.inet.tcp.delayed_ack=0
vfs.hirunningspace=5242880
vfs.lorunningspace=5242880
net.inet.tcp.local_slowstart_flightsize=65535
kern.maxfilesperproc=2048
kern.ipc.somaxconn=4096
kern.maxfiles=65536
kern.ipc.nmbclusters=65536

# Global parameters
[global]
printable = No
workgroup = NET
server string = Samba %v
comment = FreeBSD
security = share
max log size = 0
local master = No
stat cache = yes
strict sync = no
use sendfile = yes
domain master = No
read raw = yes
write raw = yes
dead time = 5
dns proxy = No
getwd cache = yes
large readwrite = yes
utmp = yes
# Not soo sold on these
nt acl support = No

# Performance Tweaks
# write cache size = 1048376
socket options = SO_RCVBUF=262144 SO_SNDBUF=262144
IPTOS_LOWDELAY TCP_NODELAY

# Audit Logging
timestamp logs = Yes
#vfs objects = extd_audit
log level = 1 vfs:2
log file = /var/log/samba/%I.log
syslog = 0
# Attempt to fix the locking issue
oplock contention limit = 5
oplock break wait time = 100

[array]
path = /home/array/
writeable = Yes
guest ok = Yes