Why is redhat slow? - Redhat

This is a discussion on Why is redhat slow? - Redhat ; Take a 4MB text file with random data in it and run an egrep or sed e.g.: time egrep -v ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \ file > /dev/null On any RH machine I've tried this on (most are dual cpu with dual ...

+ Reply to Thread
Results 1 to 14 of 14

Thread: Why is redhat slow?

  1. Why is redhat slow?

    Take a 4MB text file with random data in it and run an egrep or sed e.g.:

    time egrep -v ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    file > /dev/null

    On any RH machine I've tried this on (most are dual cpu with dual core with 2G
    ram) this takes well over 10 minutes.

    If I do the same command, same file on my slackware 10.1 machine (P3 1.2 Ghz 256MB
    ram) it only takes 1 second!

    I've noticed the same slow (over 10 minute) results on Suse and CentOS.

    sed is just as slow with the same regex.

    I took the source package that slack uses for it's grep packages and compiled them
    on RH, same result.

    Is this due to the kernels being highly boated on RH/CentOS/Suse etc?

    Note that all test subjects had stock kernels.

    Any ideas?

  2. Re: Why is redhat slow?

    In news:45f11821$0$28110$4c368faf@roadrunner.com,
    Mr. Magoo wrote:

    > Take a 4MB text file with random data in it and run an egrep or sed
    > e.g.:
    >
    > time egrep -v
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > file > /dev/null
    >
    > On any RH machine I've tried this on (most are dual cpu with dual
    > core with 2G ram) this takes well over 10 minutes.

    ....
    > Any ideas?


    Why don't you make the gzip'd text file available for others to benchmark?

  3. Re: Why is redhat slow?

    ynotssor wrote:
    >
    >
    > Why don't you make the gzip'd text file available for others to benchmark?



    OK, this won't be the exact same file and patterns will be repetitive thus not
    taking the 10 minutes like on my real data, however the time differences are still
    highly varied.

    It's not limited to just grep/sed either... everything is noticeably slower. Try a
    wc -l on a file over 200MB and compare the time it takes accross the distro's.

    Generate the test file:
    ###
    #!/bin/bash

    num=1
    >testfile

    while [ $num -lt 20665 ] ; do
    echo "blah+foo+bar_who:0.01
    blah+foo+bar_who:0.05
    blah+foo+bar_who:0.09
    blah+foo+bar_who:0.02
    blah+foo+bar_who:0.03
    blah+foo+bar_who:0.04
    blah+foo+bar_who:0.10
    blah+foo+bar_who:0.15" >> testfile
    num=`expr $num + 1`
    done
    ###

    This generates a 3.5MB file with 165312 lines.


    Here are the tests:

    ---------
    1.2Ghz Celeron 256MB RAM
    n01:# cat /etc/slackware-version
    Slackware 10.1.0

    n01:# time egrep -v \
    ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    testfile > /dev/null

    real 0m0.108s
    user 0m0.080s
    sys 0m0.020s

    ----------
    1.2Ghz PIII 1GB RAM
    g01:# cat /etc/SuSE-release
    SuSE Linux 9.3 (i586)
    VERSION = 9.3

    g01:# time egrep -v \
    ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    testfile > /dev/null

    real 4m32.997s
    user 4m32.797s
    sys 0m0.149s

    ----------
    2.4Ghz P4 1GB RAM
    p01# cat /etc/redhat-release
    Red Hat Linux release 9 (Shrike)

    p01# time egrep -v \
    ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    testfile > /dev/null

    real 2m54.955s
    user 2m51.250s
    sys 0m0.160s

  4. Re: Why is redhat slow?

    On Fri, 09 Mar 2007 03:42:21 -0600, Mr. Magoo wrote:

    > n01:# time egrep -v \
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > testfile > /dev/null



    P4 dualcore RH 4

    real 0m2.607s
    user 0m2.601s
    sys 0m0.007s


    P3 700 MHz CentOS 4

    real 0m1.756s
    user 0m1.742s
    sys 0m0.012s


  5. Re: Why is redhat slow?

    On 09 Mar 2007 in linux.redhat, Mr. Magoo wrote:

    > n01:# time egrep -v \
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > testfile > /dev/null


    FC4 on PII/400

    real 0m3.217s
    user 0m3.196s
    sys 0m0.020s

    FC6 on P4/1600

    real 0m1.110s
    user 0m1.092s
    sys 0m0.007s

    --
    Joe Makowiec
    http://makowiec.org/
    Email: http://makowiec.org/contact/?Joe

  6. Re: Why is redhat slow?

    Mr. Magoo wrote:
    > ynotssor wrote:
    >
    >>
    >>
    >> Why don't you make the gzip'd text file available for others to
    >> benchmark?

    >
    >
    >
    > OK, this won't be the exact same file and patterns will be repetitive
    > thus not taking the 10 minutes like on my real data, however the time
    > differences are still highly varied.
    >
    > It's not limited to just grep/sed either... everything is noticeably
    > slower. Try a wc -l on a file over 200MB and compare the time it takes
    > accross the distro's.
    >
    > Generate the test file:
    > ###
    > #!/bin/bash
    >
    > num=1
    > >testfile

    > while [ $num -lt 20665 ] ; do
    > echo "blah+foo+bar_who:0.01
    > blah+foo+bar_who:0.05
    > blah+foo+bar_who:0.09
    > blah+foo+bar_who:0.02
    > blah+foo+bar_who:0.03
    > blah+foo+bar_who:0.04
    > blah+foo+bar_who:0.10
    > blah+foo+bar_who:0.15" >> testfile
    > num=`expr $num + 1`
    > done
    > ###
    >
    > This generates a 3.5MB file with 165312 lines.
    >
    >
    > Here are the tests:
    >
    > ---------
    > 1.2Ghz Celeron 256MB RAM
    > n01:# cat /etc/slackware-version
    > Slackware 10.1.0
    >
    > n01:# time egrep -v \
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > testfile > /dev/null
    >
    > real 0m0.108s
    > user 0m0.080s
    > sys 0m0.020s
    >
    > ----------
    > 1.2Ghz PIII 1GB RAM
    > g01:# cat /etc/SuSE-release
    > SuSE Linux 9.3 (i586)
    > VERSION = 9.3
    >
    > g01:# time egrep -v \
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > testfile > /dev/null
    >
    > real 4m32.997s
    > user 4m32.797s
    > sys 0m0.149s
    >
    > ----------
    > 2.4Ghz P4 1GB RAM
    > p01# cat /etc/redhat-release
    > Red Hat Linux release 9 (Shrike)
    >
    > p01# time egrep -v \
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > testfile > /dev/null
    >
    > real 2m54.955s
    > user 2m51.250s
    > sys 0m0.160s


    Dear Mr. Magoo,


    RHEL ES 4U4, dual XEON 2.8 GHz with Hypertreading ON, 2GB. While running
    netscape.

    time egrep -v \
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > testfile > /dev/null


    real 0m0.502s
    user 0m0.497s
    sys 0m0.007s


    Just a question: "Why benchmark with an obsolete RH release and compare
    it with an mainstream Slackware?"

    This comparing an Oldtimer and a Porsche 948 and say the Oldtimer has
    very slow gearbox.


    Kind regards,


    Jan Gerrit Kootstra

  7. Re: Why is redhat slow?

    "Mr. Magoo" writes:

    > ynotssor wrote:
    > > Why don't you make the gzip'd text file available for others to
    > > benchmark?

    >
    >
    > OK, this won't be the exact same file and patterns will be repetitive
    > thus not taking the 10 minutes like on my real data, however the time
    > differences are still highly varied.
    >
    > It's not limited to just grep/sed either... everything is noticeably
    > slower. Try a wc -l on a file over 200MB and compare the time it takes
    > accross the distro's.


    [test file and some results snipped]

    > 2.4Ghz P4 1GB RAM
    > p01# cat /etc/redhat-release
    > Red Hat Linux release 9 (Shrike)
    >
    > p01# time egrep -v \
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > testfile > /dev/null
    >
    > real 2m54.955s
    > user 2m51.250s
    > sys 0m0.160s


    Red Hat Linux release 9 is pretty old. Back then, it was typical to have
    DMA disabled by default for IDE hard drives. Is your DMA disabled?
    (See the manual page "man hdparm") and the configuration file at
    /etc/sysconfig/harddisks.

    Scott
    --
    Scott Hemphill hemphill@alumni.caltech.edu
    "This isn't flying. This is falling, with style." -- Buzz Lightyear

  8. Re: Why is redhat slow?

    Jan Gerrit Kootstra wrote:
    >
    >
    > Dear Mr. Magoo,
    >
    >
    > RHEL ES 4U4, dual XEON 2.8 GHz with Hypertreading ON, 2GB. While running
    > netscape.
    >
    > time egrep -v \
    > > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > > testfile > /dev/null

    >
    > real 0m0.502s
    > user 0m0.497s
    > sys 0m0.007s
    >
    >
    > Just a question: "Why benchmark with an obsolete RH release and compare
    > it with an mainstream Slackware?"
    >
    > This comparing an Oldtimer and a Porsche 948 and say the Oldtimer has
    > very slow gearbox.
    >
    >
    > Kind regards,
    >
    >
    > Jan Gerrit Kootstra


    Yes I understand your point but look at the results that have been posted. range
    of 3.2 seconds and yours is fastest so far with 0.5 second.

    The slack I tested on is not latest, in fact a year or 2 old and it's results are
    consistently 0.1 second and it's much less of a machine (single P3 1.2Ghz Celeron
    256MB ram). I've also tested on Slack 8 and 9 on a PIII 533mhz with same results,
    0.1 second.

    Compound that data to 2GB with much more randomness and the 3.2 seconds to 0.1
    second ratio becomes a concern. My old clunker with slack blows away all modern
    dual core machines... that doesn't seem right.

    I had a sed last night take 4 hours on a P4 2.8 dual cpu dual core CentOS 4.4
    machine with 2GB ram, whereas on the same old P3 slack box it took 30 minutes.
    File was 3.7GB. I even tried splitting up the sed regex into 4 sed processes in
    attempt to use all 4 cpu's and still 4 hours.

    I can't post that data as it's proprietary, which is why I came up with the sample
    test above. So while not showing what I seen on real data it does show a clear
    difference in speed between them.

    I'm not trying to start a flame war on who's distro is fastest, I want to figure
    out why they are slower and fix them because I have to work on the them, and I
    dislike staying up late waiting on stuff to finish as I'm sure most all of you do
    too ;-)

    Reason I ask about kernel is slack's stock is only slightly more tuned in that you
    choose which kernel to use based on your IO... ie ide, sata, scsi with those bits
    directly compiled in, but the others are modularized with pretty much every option
    compiled as a module.

    Or could it be libraries, or how everything was compiled?

    I just want to make them as fast so I'm not waiting so long on my clients machines.

    Thanks for the replies so far and in advance for any ideas.

  9. Re: Why is redhat slow?

    hemphill@hemphills.net wrote:
    >
    > Red Hat Linux release 9 is pretty old. Back then, it was typical to have
    > DMA disabled by default for IDE hard drives. Is your DMA disabled?
    > (See the manual page "man hdparm") and the configuration file at
    > /etc/sysconfig/harddisks.
    >
    > Scott


    All boxes in my test have scsi drives.

  10. Re: Why is redhat slow?

    Mr. Magoo wrote:
    > Jan Gerrit Kootstra wrote:
    >
    >>
    >>
    >> Dear Mr. Magoo,
    >>
    >>
    >> RHEL ES 4U4, dual XEON 2.8 GHz with Hypertreading ON, 2GB. While
    >> running netscape.
    >>
    >> time egrep -v \
    >> > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    >> > testfile > /dev/null

    >>
    >> real 0m0.502s
    >> user 0m0.497s
    >> sys 0m0.007s
    >>
    >>
    >> Just a question: "Why benchmark with an obsolete RH release and
    >> compare it with an mainstream Slackware?"
    >>
    >> This comparing an Oldtimer and a Porsche 948 and say the Oldtimer has
    >> very slow gearbox.
    >>
    >>
    >> Kind regards,
    >>
    >>
    >> Jan Gerrit Kootstra

    >
    >
    > Yes I understand your point but look at the results that have been
    > posted. range
    > of 3.2 seconds and yours is fastest so far with 0.5 second.
    >
    > The slack I tested on is not latest, in fact a year or 2 old and it's
    > results are
    > consistently 0.1 second and it's much less of a machine (single P3
    > 1.2Ghz Celeron
    > 256MB ram). I've also tested on Slack 8 and 9 on a PIII 533mhz with same
    > results,
    > 0.1 second.
    >
    > Compound that data to 2GB with much more randomness and the 3.2 seconds
    > to 0.1
    > second ratio becomes a concern. My old clunker with slack blows away all
    > modern
    > dual core machines... that doesn't seem right.
    >
    > I had a sed last night take 4 hours on a P4 2.8 dual cpu dual core
    > CentOS 4.4
    > machine with 2GB ram, whereas on the same old P3 slack box it took 30
    > minutes.
    > File was 3.7GB. I even tried splitting up the sed regex into 4 sed
    > processes in
    > attempt to use all 4 cpu's and still 4 hours.
    >
    > I can't post that data as it's proprietary, which is why I came up with
    > the sample
    > test above. So while not showing what I seen on real data it does show a
    > clear
    > difference in speed between them.
    >
    > I'm not trying to start a flame war on who's distro is fastest, I want
    > to figure
    > out why they are slower and fix them because I have to work on the them,
    > and I
    > dislike staying up late waiting on stuff to finish as I'm sure most all
    > of you do
    > too ;-)
    >
    > Reason I ask about kernel is slack's stock is only slightly more tuned
    > in that you
    > choose which kernel to use based on your IO... ie ide, sata, scsi with
    > those bits
    > directly compiled in, but the others are modularized with pretty much
    > every option
    > compiled as a module.
    >
    > Or could it be libraries, or how everything was compiled?
    >
    > I just want to make them as fast so I'm not waiting so long on my
    > clients machines.
    >
    > Thanks for the replies so far and in advance for any ideas.

    Hello Mr. Magoo,


    Sorry for my strong response.

    I am not confished myself that dual core processors are the best
    hardware choice at the moment, but cannot prove it. I have no dual core
    hardware to run tests on.


    Kind regards,


    Jan Gerrit

    P.S. If you reply to my personal mailbox, then I would like you to use a
    non-blocked email-address. If that is against your policies, then only
    reply to the newsgroup.

  11. Re: Why is redhat slow?

    Jan Gerrit Kootstra wrote:
    >
    > Hello Mr. Magoo,
    >
    >
    > Sorry for my strong response.
    >
    > I am not confished myself that dual core processors are the best
    > hardware choice at the moment, but cannot prove it. I have no dual core
    > hardware to run tests on.
    >
    >
    > Kind regards,
    >
    >
    > Jan Gerrit
    >
    > P.S. If you reply to my personal mailbox, then I would like you to use a
    > non-blocked email-address. If that is against your policies, then only
    > reply to the newsgroup.



    Hi Jan,


    No worries.

    Sorry about sending directly to you, I accidentally hit reply rather than reply to
    group.


    Regards

  12. Re: Why is redhat slow?

    Mr. Magoo wrote:
    > It's not limited to just grep/sed either... everything is noticeably
    > slower. Try a wc -l on a file over 200MB and compare the time it takes
    > accross the distro's.


    When did RedHat switch to using UTF-8 as the default character encoding?
    If RHL9 uses UTF-8, this could be the reason. Older versions of GNU
    text utilities had performance problems with Unicode text.

    --
    Markku Kolkka
    markku.kolkka@iki.fi



  13. Re: Why is redhat slow?

    Mr. Magoo wrote:

    > ynotssor wrote:
    >>
    >>
    >> Why don't you make the gzip'd text file available for others to
    >> benchmark?

    >
    >
    > OK, this won't be the exact same file and patterns will be repetitive thus
    > not taking the 10 minutes like on my real data, however the time
    > differences are still highly varied.
    >
    > It's not limited to just grep/sed either... everything is noticeably
    > slower. Try a wc -l on a file over 200MB and compare the time it takes
    > accross the distro's.
    >
    > Generate the test file:
    > ###
    > #!/bin/bash
    >
    > num=1
    > >testfile

    > while [ $num -lt 20665 ] ; do
    > echo "blah+foo+bar_who:0.01
    > blah+foo+bar_who:0.05
    > blah+foo+bar_who:0.09
    > blah+foo+bar_who:0.02
    > blah+foo+bar_who:0.03
    > blah+foo+bar_who:0.04
    > blah+foo+bar_who:0.10
    > blah+foo+bar_who:0.15" >> testfile
    > num=`expr $num + 1`
    > done
    > ###
    >
    > This generates a 3.5MB file with 165312 lines.
    >
    >
    > Here are the tests:
    >
    > ---------
    > 1.2Ghz Celeron 256MB RAM
    > n01:# cat /etc/slackware-version
    > Slackware 10.1.0
    >
    > n01:# time egrep -v \
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > testfile > /dev/null
    >
    > real 0m0.108s
    > user 0m0.080s
    > sys 0m0.020s
    >
    > ----------
    > 1.2Ghz PIII 1GB RAM
    > g01:# cat /etc/SuSE-release
    > SuSE Linux 9.3 (i586)
    > VERSION = 9.3
    >
    > g01:# time egrep -v \
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > testfile > /dev/null
    >
    > real 4m32.997s
    > user 4m32.797s
    > sys 0m0.149s
    >
    > ----------
    > 2.4Ghz P4 1GB RAM
    > p01# cat /etc/redhat-release
    > Red Hat Linux release 9 (Shrike)
    >
    > p01# time egrep -v \
    > ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06|0.07" \
    > testfile > /dev/null
    >
    > real 2m54.955s
    > user 2m51.250s
    > sys 0m0.160s


    My results on a 2.4GHz P4, RHEL4ES on VmWare Workstation with 384MB:

    time egrep -v ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06
    0.07" testfile > /dev/null

    real 0m0.740s
    user 0m0.629s
    sys 0m0.035s

    Sander


  14. Re: Why is redhat slow?


    >
    > My results on a 2.4GHz P4, RHEL4ES on VmWare Workstation with 384MB:
    >
    > time egrep -v ":$|:0$|0\.0$|:0\.00|0\.01|:0\.$|0.02|0.03|0.04|0.0 5|0.06
    > 0.07" testfile > /dev/null
    >
    > real 0m0.740s
    > user 0m0.629s
    > sys 0m0.035s
    >
    > Sander


    Just for fun one more: same 2.4GZ P4, Slack 11.0 on VmWare, 32MB:

    real 0m0.081s
    user 0m0.070s
    sys 0m0.010s

    Sander

+ Reply to Thread