Samba performance tuning - Fedora Core 4 and Windows XP - SMB

This is a discussion on Samba performance tuning - Fedora Core 4 and Windows XP - SMB ; Here is my setup: Linux server: - Supermicro X6DA8-G2 mainboard - Dual Intel Xeon 3.6 GHz, 800 MHz FSB - 8GB DDR2 PC3200 ECC memory - 2 x 300GB 10.000RPM SCSI U320 disks (Software RAID-1 via mdadm tool) - Gigabit ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: Samba performance tuning - Fedora Core 4 and Windows XP

  1. Samba performance tuning - Fedora Core 4 and Windows XP

    Here is my setup:

    Linux server:
    - Supermicro X6DA8-G2 mainboard
    - Dual Intel Xeon 3.6 GHz, 800 MHz FSB
    - 8GB DDR2 PC3200 ECC memory
    - 2 x 300GB 10.000RPM SCSI U320 disks (Software RAID-1 via mdadm tool)
    - Gigabit LAN on mainboard
    - Fedora Core 4, 2.6.11-1.1369_FC4smp kernel
    - SAMBA server version 3.0.14a-2
    - LAN MTU set to 1500 (reported by ifconfig eth0)

    Windows XP client:
    - Asus P4C800-E mainboard
    - Intel P4 3.4GHz CPU
    - 2GB DDR1 PC3200 memory
    - 2 x WD Raptor 74GB 10.000RPM SATA disks (RAID-0 via ICH5R chip on
    mainboard)
    - Gigabit LAN on mainboard
    - Windows XP SP2
    - LAN MTU set to 1500 (via DRTCP utility)

    smb.conf file:
    # Samba config file created using SWAT
    # from 192.168.123.100 (192.168.123.100)
    # Date: 2005/12/28 13:12:24
    # Global parameters
    [global]
    workgroup = TEST1
    server string = Samba Server
    username map = /etc/samba/smbusers
    log file = /var/log/samba/%m.log
    max log size = 50
    max xmit = 65535
    socket options = IPTOS_LOWDELAY TCP_NODELAY SO_RCVBUF=262140
    SO_SNDBUF=262140
    printcap name = /etc/printcap
    dns proxy = No
    ldap ssl = no
    idmap uid = 16777216-33554431
    idmap gid = 16777216-33554431
    guest ok = Yes
    cups options = raw
    [public]
    comment = public directory: R/O by all
    path = /data/d0/disk0/public


    When I FTP an about 5.2 GB test.tar.gz file from the Linux server to
    the Windows XP client, ftp reports about 44MB/sec transfer speed.

    How can I tune further the transfer speed?

    My network switches don't support jumbo frames, so that is not an
    option at the moment. Also I haven't changed any networking parameters
    on the Windows XP side yet, except set MTU to 1500 with DRTCP utility.
    What tuning would help on the client side?

    Finally is there a way to FTP from the Linux server, but writing to
    /dev/null on the Windows XP side in order to eliminate the overhead of
    the disk-subsystem on the client side, sort of a dd over the LAN?

    dd on the Linux server reports that the disk-subsystem is capable of
    about 90MB/sec, so there is room to improve the LAN transfer rate (I
    hope).

    Any ideas welcome.

    TIA
    Jeff


  2. Re: Samba performance tuning - Fedora Core 4 and Windows XP

    jeff000069@yahoo.com wrote:
    ....
    > When I FTP an about 5.2 GB test.tar.gz file from the Linux server to
    > the Windows XP client, ftp reports about 44MB/sec transfer speed.


    FTP? Samba is not involved when you ftp.

    What does top say on your linux box when you transfer
    this file? What is especially interesting is the Cpu line
    at the top, how much time is spent in "sy"? If this is high,
    you might need a better NIC.

    You could try and test your raw network speed with ttcp (you'll
    have to ask google where you can find this for Windows).


    ....
    > Finally is there a way to FTP from the Linux server, but writing to
    > /dev/null on the Windows XP side in order to eliminate the overhead of
    > the disk-subsystem on the client side, sort of a dd over the LAN?


    If you actually mean ftp, you can do a:

    get test.tar.gz nul:

    in the ftp program on the Windows client.

    Mogens

    --
    Mogens Kjær, Dataarkæolog
    Email: mk@datamuseum.dk
    Homepage: http://www.datamuseum.dk

  3. Re: Samba performance tuning - Fedora Core 4 and Windows XP

    Hej Mogens,

    Thanks for replying.

    I did as you suggested on the Windows XP: "get test.tar.gz nul:"
    This gives a transfer rate of 90-95MB/sec, which sounds too good to be
    true.
    Is such a rate even possible on GigE without jumbo frames? (I will have
    to google for maximum possible throughput of GigE).

    While the above get was running on the client, top on the server shows
    0.5%. I suppose my NIC is good enough?

    I will also try with TTCP (once I find a version for XP).

    Jeff.


  4. Re: Samba performance tuning - Fedora Core 4 and Windows XP

    jeff000069@yahoo.com wrote:
    > Hej Mogens,
    >
    > Thanks for replying.
    >
    > I did as you suggested on the Windows XP: "get test.tar.gz nul:"
    > This gives a transfer rate of 90-95MB/sec, which sounds too good to be
    > true.
    > Is such a rate even possible on GigE without jumbo frames? (I will have
    > to google for maximum possible throughput of GigE).


    I get the same rate, also without jumbo frames.

    ttcp is a bit higher, around 100Mbytes/sec.

    >
    > While the above get was running on the client, top on the server shows
    > 0.5%. I suppose my NIC is good enough?


    I doubt that any NIC can feed a gigabit network
    with a load of just 0.5 %. What exactly does "top" show
    during the transfer?

    Mogens

    --
    Mogens Kjær, Dataarkæolog
    Email: mk@datamuseum.dk
    Homepage: http://www.datamuseum.dk

  5. Re: Samba performance tuning - Fedora Core 4 and Windows XP


    I looked at output of top again on the server (while a xcopy on Windows
    XP was copying the test.tar.gz file from the server). Typical values
    are:
    0.3%us 2.5%sy 0.0%ni 96.0%id 0.2%wa 0.0%hi 0.7%si

    The numbers are of course the average of both Xeon processors (with
    hyperthreading enabled). The linux server is not running anything else
    at the moment, besides standard OS processes.

    The 0.5% mentioned in my earlier post was the "sy" value while running
    a FTP get on the Windows XP. I should probably rerun this just to be
    sure, since it does sound a bit low.

    Incidently I forget to mention the results of the CIFS throughput in my
    initial post, here they are:
    copy U:\test.tar.gz to C:\ gives 42MB/sec
    >From GUI copy and paste to C:\ gives 35MB/sec


    Keep in mind that FTP gave about 44MB/sec.
    Using the Windows XP GUI (copy & paste) gave surprisingly low result of
    35MB/sec.

    Is there a way to cache files on the Windows XP side, to avoid reading
    files over-the-wire all the time?

    I will be storing all my digital photos on the Linux SAMBA server and
    viewing these from Windows XP via applications such as Photoshop or
    BreezeBrowser. Since I have 2GB memory on my XP client, caching the
    read-only photos would be usefull.

    Thanks again for all your help.

    Jeff.


  6. Re: Samba performance tuning - Fedora Core 4 and Windows XP

    jeff000069@yahoo.com wrote:
    > I looked at output of top again on the server (while a xcopy on Windows
    > XP was copying the test.tar.gz file from the server). Typical values
    > are:
    > 0.3%us 2.5%sy 0.0%ni 96.0%id 0.2%wa 0.0%hi 0.7%si
    >
    > The numbers are of course the average of both Xeon processors (with
    > hyperthreading enabled). The linux server is not running anything else
    > at the moment, besides standard OS processes.


    On my server (2xXeon 2.4 GHz with HT) and an Intel server NIC,
    I get 100% system CPU usage (i.e. one of the four CPU's is totally
    busy) when I do a ttcp. What kind of NIC do you have on your system?

    ....
    > Is there a way to cache files on the Windows XP side, to avoid reading
    > files over-the-wire all the time?


    Isn't this what oplocks are used for in samba?

    - I always turn these off...

    Mogens

    --
    Mogens Kjær, Dataarkæolog
    Email: mk@datamuseum.dk
    Homepage: http://www.datamuseum.dk

  7. Re: Samba performance tuning - Fedora Core 4 and Windows XP

    I redid the test to be sure about the CPU load.

    While xcopy command on Windows XP was copying the test.tar.gz file from
    the Samba server, I looked at the graphical output of
    /usr/bin/gnome-system-monitor. This shows CPU4 at more or less 15%
    constant load during the entire duration of the data transfer. The
    other 3 CPUs are at virtually zero.

    Clearly the top output is wrong, it only shows 1 line for all the CPUs.
    I was wrongly assumming that this was the average of all 4 CPUs, but
    seems more likely to be CPU1 only. Assumption is the mother of all
    screw-ups, as the saying goes....I need surely to get a better version
    of top than the included version in FC4.

    My NIC is the onboard port of my Supermicro X6DA8-G2 mainboard, this is
    based on the Intel 82546GB dual port Gigabit controller chip. It
    appears that the FC4 drivers for this NIC are performing well.

    Would moving to jumbo frames be usefull in your opinion? (I would have
    to buy a new switch, of course).

    Thanks for the pointer to oplocks in SAMBA, I will surely research
    further. I am interested in caching since my SAMBA server is mainly
    serving read-only content (mainly digital photos).

    Other ideas surely welcome!

    Jeff


  8. Re: Samba performance tuning - Fedora Core 4 and Windows XP

    jeff000069@yahoo.com wrote:
    ....
    > Clearly the top output is wrong, it only shows 1 line for all the CPUs.
    > I was wrongly assumming that this was the average of all 4 CPUs, but
    > seems more likely to be CPU1 only. Assumption is the mother of all
    > screw-ups, as the saying goes....I need surely to get a better version
    > of top than the included version in FC4.


    You can press "1" while top is running, and you'll get a line
    for each cpu.

    >
    > My NIC is the onboard port of my Supermicro X6DA8-G2 mainboard, this is
    > based on the Intel 82546GB dual port Gigabit controller chip. It
    > appears that the FC4 drivers for this NIC are performing well.


    I have a 82545GM controller. This is on a redhat 9 machine, with
    a (more or less) vanilla 2.4.32 kernel.

    >
    > Would moving to jumbo frames be usefull in your opinion? (I would have
    > to buy a new switch, of course).


    I have no experience with jumbo frames. None of my switches support them.

    Mogens

    --
    Mogens Kjaer, Carlsberg A/S, Computer Department
    Gamle Carlsberg Vej 10, DK-2500 Valby, Denmark
    Phone: +45 33 27 53 25, Fax: +45 33 27 47 08
    Email: mk@crc.dk Homepage: http://www.crc.dk

  9. Re: Samba performance tuning - Fedora Core 4 and Windows XP

    I try to summarise my results/obervations so far, after admittedly some
    quick & dirty testing:

    TEST1: From Client1 (Windows XP, Asus P4C800-ED, Intel P4 3.4GHz, 2GB
    RAM, RAID-0 stripe on 2 x 74GB raptors):
    run1: ftp U:\test.tar.gz C:\ gives 45-55MB/sec & CPU-util=85%
    run2: ftp U:\test.tar.gz nul: gives 90-100MB/sec & CPU-util=100%
    run3: copy U:\test.tar.gz C:\ gives 40-45MB/sec & CPU-util=55%
    run4: copy U:\test.tar.gz NUL gives 55-60MB/sec & CPU-util=35%

    TEST2: From Client2 (Windows XP, Asus A8N-SLI Premium, AMD Athlon 3500,
    1GB RAM, single 74GB raptor):
    run1: ftp U:\test.tar.gz C:\ gives 50-55MB/sec & CPU-util=55%
    run2: ftp U:\test.tar.gz nul: gives 65-70MB/sec & CPU-util=90%
    run3: copy U:\test.tar.gz C:\ gives 50-55/sec & CPU-util=60%
    run4: copy U:\test.tar.gz NUL gives 75-80/sec & CPU-util=60%

    Client1 has an integrated Intel Pro/1000 CT NIC and Client2 has an
    integrated Nvidia nForce4-SLI NIC. Client1 network utilisation improved
    significantly after upgrading to latest network drivers from Intel
    website.

    Each run was executed 4 times, since I noticed significant better
    transfer rates on the 4th run; appears that file-system caching of the
    5.2GB test file really kicked-in on the server side after a few runs.

    On client1, going from ftp to copy (i.e. using the CIFS layer) incurred
    quite a penalty. This was not the case on client2; I do not know why?

    On client2, copy is faster than ftp, when destination is NUL. A bit
    strange?

    It is clear that approaching gigE wire speed requires close to 100%
    CPU-utilisation, on the above 2 clients. (Pity I do not have a dual-cpu
    or dual-core client I could run the test with). A thanks to Mogens who
    pointed this out.

    On the server only about 15% of CPU4 is required, according to
    gnome-system-monitor. I wonder how accurate this is, but anyway there
    is sufficient cpu power on the server so it doesn't really matter.
    Running dd if=test.tar.gz of=/dev/zero bs=4K on the server gave
    100MB/sec and even 200MB/sec once file-system caching kicked-in.
    Increasing bs to 128K even gave 1.2GB/sec!!! Clearly the server is
    powerfull enough to provide gigE wire speeds to clients without
    choking.

    The Intel NIC appears to have the advantage relative to the Nvidia NIC.
    It was possible to reach 100MB/sec on client1, but never on client2
    when executing run2. I am not sure if this is hardware or software
    driver related, perhaps a combination of both?

    Since gigE wire speed is being reached (on some tests), jumbo frames
    probably won't help except for reducing cpu utilisation.

    Tentative Conclusion:
    1) My next client PC should have at least have a dual-core cpu (2 x
    dual-cores would be even better for Photoshop, does Windows XP support
    4 CPUs?).
    2) Intel NICs are the preffered choice, but I should probably run more
    tests to be really sure on this point.
    3) The SCSI disk sub-system on the server is superior to the SATA disk
    sub-systems on the clients. But this is probably due to the faster SCSI
    disks, which have about half the seek time of the SATA raptor disks.

    Hope above is of help to someone.

    Jeff


+ Reply to Thread