Linux file copy slowness ?? - Redhat

This is a discussion on Linux file copy slowness ?? - Redhat ; Hi, We are trying to copy a 15Gig file from one directory to another (inside same partition) for example: cp /data/BigFile /data/data/ (Bigfile - being file that is greater than 15 gig). It takes about 32 minutes to finish copying. ...

+ Reply to Thread
Results 1 to 8 of 8

Thread: Linux file copy slowness ??

  1. Linux file copy slowness ??

    Hi,

    We are trying to copy a 15Gig file from one directory to another
    (inside same partition) for example:

    cp /data/BigFile /data/data/ (Bigfile - being file that is greater
    than 15 gig).

    It takes about 32 minutes to finish copying. When we do the same
    operation on a windows 2003 server it takes almost HALF the time.

    I am wondering if this behaviour is normal or something wrong with
    system?kernel? etc.

    The system is a hefty: Dell 1850 server with Dual Xeons 3.2 ghz (hyper
    threaded) with 2 Gig ram, SCSI hard drives 10k rpm.

    Also, while the system is copying the file, I don't see any kernel
    errors or IO errros and all the time while the sytem is copying this
    drive the cpu usage is at 50 Any help is highly appreciated.

    Forgot to mention on another post: The file system is EXT3 and OS
    -Redhat Enterprise 3

    -Parvinder Bhasin


  2. Re: Linux file copy slowness ??

    parvinderb@gmail.com writes:
    >Hi,
    >
    >We are trying to copy a 15Gig file from one directory to another
    >(inside same partition) for example:
    >
    >cp /data/BigFile /data/data/ (Bigfile - being file that is greater
    >than 15 gig).


    So, both files are on the same disk, and you're getting 7.8Mb/second.

    For reference, this is the output from a 1gb random transfer on
    a 36MB 10krpm seagate cheetah:

    $ t2 -r -f /dev/sday -b 131072 -i 1000
    Device /dev/sday has 70022843 512-byte blocks
    131072000 bytes (1000 131072-byte blocks to 1 devs) in 14.50 seconds (9039.45 Kbytes/sec)

    (-r accesses the device randomly).

    Drop the block size, and:
    $ t2 -r -f /dev/sday -b 4096 -i 1000
    Device /dev/sday has 70022843 512-byte blocks
    4096000 bytes (1000 4096-byte blocks to 1 devs) in 9.00 seconds (455.11 Kbytes/sec)

    So, your 7.8MB/second seems pretty good. You may get better
    performance if you mk2efs your filesystem with -b 8196 instead
    of 4096.

    $ stat -f /
    File: "/"
    ID: 0 Namelen: 255 Type: ext2/ext3
    Blocks: Total: 10078499 Free: 9590416 Available: 9078445 Size: 4096
    Inodes: Total: 5128192 Free: 5065884

    (the block size is the "Size:" field).

    -----------Just for reference:

    Pure sequential access on a Cheetah:
    $ t2 -f /dev/sday -b 131072 -i 1000
    Device /dev/sday has 70022843 512-byte blocks
    131072000 bytes (1000 131072-byte blocks to 1 devs) in 3.50 seconds (37449.14 Kbytes/sec)

    And, the advantages of fibrechannel and lots of JBOD (and a 2Gbit switch):

    $ t2 -f /dev/sdag -f /dev/sdah -f /dev/sdam -f /dev/sdan -f /dev/sdaq -f /dev/sdap -f /dev/sdaa -f /dev/sdab -b 131072 -i 10000
    Device /dev/sdag has 70022843 512-byte blocks
    Device /dev/sdah has 70022843 512-byte blocks
    Device /dev/sdam has 70022843 512-byte blocks
    Device /dev/sdan has 70022843 512-byte blocks
    Device /dev/sdaq has 163397632 512-byte blocks
    Device /dev/sdap has 163397632 512-byte blocks
    Device /dev/sdaa has 163397632 512-byte blocks
    Device /dev/sdab has 163397632 512-byte blocks
    10485760000 bytes (10000 131072-byte blocks to 8 devs) in 52.00 seconds (201649.23 Kbytes/sec)


    Note that the test program, t2, uses O_DIRECT to bypass the operating system
    file cache, and uses io_submit(2) fpr asynchronous I/O.

    scott

  3. Re: Linux file copy slowness ??

    parvinderb@gmail.com wrote:
    > Hi,
    >
    > We are trying to copy a 15Gig file from one directory to another (inside
    > same partition) for example:
    >
    > cp /data/BigFile /data/data/ (Bigfile - being file that is greater than
    > 15 gig).
    >
    > It takes about 32 minutes to finish copying. When we do the same
    > operation on a windows 2003 server it takes almost HALF the time.
    >
    > I am wondering if this behaviour is normal or something wrong with
    > system?kernel? etc.
    >
    > The system is a hefty: Dell 1850 server with Dual Xeons 3.2 ghz (hyper
    > threaded) with 2 Gig ram, SCSI hard drives 10k rpm.
    >
    > Also, while the system is copying the file, I don't see any kernel errors
    > or IO errros and all the time while the sytem is copying this drive the
    > cpu usage is at 50 Any help is highly appreciated.
    >
    > Forgot to mention on another post: The file system is EXT3 and OS
    > -Redhat Enterprise 3
    >

    Do you have another scsi drive (preferably, but not necessarily on a
    different SCSI controller) on the system? If so, it might be faster to copy
    to a spare spot on a different drive and then copy it back. This would cut
    the seek time dramatically if that is the bottleneck, as it very well might be.

    But I have 10,000 rpm hard drives on an Ultra/320 SCSI controller and I have
    seen 50MBytes/second transfer rate on working system (also RHEL3) for a
    sustained interval (many seconds), but with a full load of other processes
    running, messing up the L3 cache on the dual 3.06GHz hyperthreades Xeons,
    the disk IO rate can drop to under 15 MBytes/second.

    One trick is to ask how you do the copying. If you use _cpio --pass-through_
    it might run faster than with the _cp_ command.

    --
    .~. Jean-David Beyer Registered Linux User 85642.
    /V\ PGP-Key: 9A2FC99A Registered Machine 241939.
    /( )\ Shrewsbury, New Jersey http://counter.li.org
    ^^-^^ 20:50:00 up 8 days, 11:36, 4 users, load average: 4.38, 4.27, 4.14

  4. Re: Linux file copy slowness ??

    parvinderb@gmail.com wrote:
    >
    >
    > It takes about 32 minutes to finish copying. When we do the same
    > operation on a windows 2003 server it takes almost HALF the time.
    >
    > I am wondering if this behaviour is normal or something wrong with
    > system?kernel? etc.
    >
    >

    I'm not an expert so don't know if this is relevant, but I'll throw it
    in just in case.

    A while ago I was playing with multibooting XP and core 2 each on their
    own hardrive, I was selecting OS with bios, somehow I managed to turn
    the CPU cache off. This crippled XP slowed it down by a factor of about
    1,000, took 30minutes just to boot!!, but core 2 seemed to be unafected.
    Obviously XP makes a great use of cpu cache and fedora doesn't, would
    this affect file juggling speeds???

  5. Re: Linux file copy slowness ??

    decrepit <""mikelayzell\"@netscape .net"> writes:
    >parvinderb@gmail.com wrote:
    >>
    >>
    >> It takes about 32 minutes to finish copying. When we do the same
    >> operation on a windows 2003 server it takes almost HALF the time.
    >>
    >> I am wondering if this behaviour is normal or something wrong with
    >> system?kernel? etc.
    >>
    >>

    >I'm not an expert so don't know if this is relevant, but I'll throw it
    >in just in case.
    >
    >A while ago I was playing with multibooting XP and core 2 each on their
    >own hardrive, I was selecting OS with bios, somehow I managed to turn
    >the CPU cache off. This crippled XP slowed it down by a factor of about
    >1,000, took 30minutes just to boot!!, but core 2 seemed to be unafected.
    >Obviously XP makes a great use of cpu cache and fedora doesn't, would
    >this affect file juggling speeds???



    "Obviously?" Hardly. Fedora core 2 (or any linux for that matter)
    will use the processor cache just as much as Windows. The lack of
    the cache will slow both down quite significantly. If your installation
    of FC2 ran fast, then the cache was enabled.

    It's possible that Windows obeyed the BIOS setting, and linux didn't, but
    that's a different issue.

    scott

  6. Re: Linux file copy slowness ??

    Scott Lurndal wrote:

    > "Obviously?" Hardly. Fedora core 2 (or any linux for that matter)
    > will use the processor cache just as much as Windows. The lack of
    > the cache will slow both down quite significantly. If your installation
    > of FC2 ran fast, then the cache was enabled.
    >
    > It's possible that Windows obeyed the BIOS setting, and linux didn't, but
    > that's a different issue.
    >
    > scott


    Thanks Scott for putting me right, I guess I didn't consider that linux
    wouldn't take any notice of bios.
    So my point doesn't help this thread, but I've learnt something!
    Mike

  7. Re: Linux file copy slowness ??

    Scott Lurndal wrote:

    > "Obviously?" Hardly. Fedora core 2 (or any linux for that matter) will
    > use the processor cache just as much as Windows. The lack of the cache
    > will slow both down quite significantly. If your installation of FC2
    > ran fast, then the cache was enabled.
    >
    > It's possible that Windows obeyed the BIOS setting, and linux didn't, but
    > that's a different issue.
    >

    How do I tell if Linux obeys my BIOS setting? In the BIOS, I get to decide
    if it is to use the L3 cache or not. I got the idea that the kernel got no
    choice in the matter. On my machine, I have two hyperthreaded 3.06 GHz Xeon
    processors. These have

    /proc/cpuinfo says, in part (there are four of these, but you get the idea).

    $ cat /proc/cpuinfo
    processor : 0
    vendor_id : GenuineIntel
    cpu family : 15
    model : 2
    model name : Intel(R) Xeon(TM) CPU 3.06GHz
    stepping : 5
    cpu MHz : 3056.546
    cache size : 512 KB
    physical id : 0
    siblings : 2
    runqueue : 0
    fdiv_bug : no
    hlt_bug : no
    f00f_bug : no
    coma_bug : no
    fpu : yes
    fpu_exception : yes
    cpuid level : 2
    wp : yes
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
    cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
    bogomips : 6094.84

    Now in these processors, there are "three" caches, as shown by /var/log/dmesg:

    CPU: Trace cache: 12K uops, L1 D cache: 8K
    CPU: L2 cache: 512K
    CPU: L3 cache: 1024K

    It seems to me that the kernel is aware of the L3 cache, but it does not
    appear in /proc/cpuinfo. So does the kernel use it or not? I always assumed
    it did.

    --
    .~. Jean-David Beyer Registered Linux User 85642.
    /V\ PGP-Key: 9A2FC99A Registered Machine 241939.
    /( )\ Shrewsbury, New Jersey http://counter.li.org
    ^^-^^ 07:40:00 up 11 days, 22:26, 3 users, load average: 4.38, 4.26, 4.20

  8. Re: Linux file copy slowness ??

    Jean-David Beyer writes:
    >Scott Lurndal wrote:
    >
    >> "Obviously?" Hardly. Fedora core 2 (or any linux for that matter) will
    >> use the processor cache just as much as Windows. The lack of the cache
    >> will slow both down quite significantly. If your installation of FC2
    >> ran fast, then the cache was enabled.
    >>
    >> It's possible that Windows obeyed the BIOS setting, and linux didn't, but
    >> that's a different issue.
    >>

    >How do I tell if Linux obeys my BIOS setting? In the BIOS, I get to decide
    >if it is to use the L3 cache or not. I got the idea that the kernel got no
    >choice in the matter. On my machine, I have two hyperthreaded 3.06 GHz Xeon
    >processors. These have
    >
    >/proc/cpuinfo says, in part (there are four of these, but you get the idea).
    >
    >$ cat /proc/cpuinfo
    >processor : 0
    >vendor_id : GenuineIntel
    >cpu family : 15
    >model : 2
    >model name : Intel(R) Xeon(TM) CPU 3.06GHz
    >stepping : 5
    >cpu MHz : 3056.546
    >cache size : 512 KB
    >physical id : 0
    >siblings : 2
    >runqueue : 0
    >fdiv_bug : no
    >hlt_bug : no
    >f00f_bug : no
    >coma_bug : no
    >fpu : yes
    >fpu_exception : yes
    >cpuid level : 2
    >wp : yes
    >flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
    >cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
    >bogomips : 6094.84
    >
    >Now in these processors, there are "three" caches, as shown by /var/log/dmesg:
    >
    >CPU: Trace cache: 12K uops, L1 D cache: 8K
    >CPU: L2 cache: 512K
    >CPU: L3 cache: 1024K
    >
    >It seems to me that the kernel is aware of the L3 cache, but it does not
    >appear in /proc/cpuinfo. So does the kernel use it or not? I always assumed
    >it did.


    it really has no choice. Weather or not the L3 cache
    is used is governed by a set of bits in a configuration register associated
    with the memory controller/northbridge chipset.

    The BIOS is generally the only code which manipulates these configuration
    registers, as they are highly chipset dependent.


    L2 cache settings are somewhat under the control of the operating system
    via the architecturally defined MTRR registers and attribute bits in the page tables.

    scott

    >
    >--
    > .~. Jean-David Beyer Registered Linux User 85642.
    > /V\ PGP-Key: 9A2FC99A Registered Machine 241939.
    > /( )\ Shrewsbury, New Jersey http://counter.li.org
    > ^^-^^ 07:40:00 up 11 days, 22:26, 3 users, load average: 4.38, 4.26, 4.20


+ Reply to Thread