Speed up sed command - Unix

This is a discussion on Speed up sed command - Unix ; I hope this will be a quick one. I have a script for picking out several lines of data. When it gets to the sed command it knows what line number to start on and print every line untill you ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: Speed up sed command

  1. Speed up sed command

    I hope this will be a quick one.

    I have a script for picking out several lines of data. When it gets to
    the sed command it knows what line number to start on and print every
    line untill you come to the blank line. The sed command works correctly
    however it is slow because the log file can be 1 or 2 gig. Now for it
    to grab one record takes about 2-3 seconds. But in one log file I may
    need 1000 records. So on each loop It is reading the entire file. Is
    there a faster method while still using sed where I can make it a one
    loop script or is it time to break out the perl.

    [Below is the command I use.]
    sed -n "11353,/^$/p" filename.log


  2. Re: Speed up sed command

    On 2005-08-08, mp34ken@hotmail.com wrote:
    > I hope this will be a quick one.
    >
    > I have a script for picking out several lines of data. When it gets to
    > the sed command it knows what line number to start on and print every
    > line untill you come to the blank line. The sed command works correctly
    > however it is slow because the log file can be 1 or 2 gig. Now for it
    > to grab one record takes about 2-3 seconds. But in one log file I may
    > need 1000 records. So on each loop It is reading the entire file. Is
    > there a faster method while still using sed where I can make it a one
    > loop script or is it time to break out the perl.
    >
    > [Below is the command I use.]
    > sed -n "11353,/^$/p" filename.log


    I'd use awk:

    awk 'NR >= 11353 {print; if ( NF == 0 ) exit }'

    --
    Chris F.A. Johnson
    ================================================== ================
    Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress


  3. Re: Speed up sed command

    In article <1123528079.173294.80070@g43g2000cwa.googlegroups.c om>,
    wrote:
    >I hope this will be a quick one.
    >
    >I have a script for picking out several lines of data. When it gets to
    >the sed command it knows what line number to start on and print every
    >line untill you come to the blank line. The sed command works correctly
    >however it is slow because the log file can be 1 or 2 gig. Now for it
    >to grab one record takes about 2-3 seconds. But in one log file I may
    >need 1000 records. So on each loop It is reading the entire file. Is
    >there a faster method while still using sed where I can make it a one
    >loop script or is it time to break out the perl.
    >
    >[Below is the command I use.]
    >sed -n "11353,/^$/p" filename.log
    >


    Try:

    sed -n -e '11353,/^$/p' -e '114,/^$/p' -e '453353,/^$/p' filename.log

    I don't know what will happen if the command line becomes too long.

    --
    rr

  4. Re: Speed up sed command

    On 8 Aug 2005 12:07:59 -0700, mp34ken@hotmail.com wrote:
    > I hope this will be a quick one.
    >
    > I have a script for picking out several lines of data. When it gets to
    > the sed command it knows what line number to start on and print every
    > line untill you come to the blank line. The sed command works correctly
    > however it is slow because the log file can be 1 or 2 gig. Now for it
    > to grab one record takes about 2-3 seconds. But in one log file I may
    > need 1000 records. So on each loop It is reading the entire file. Is
    > there a faster method while still using sed where I can make it a one
    > loop script or is it time to break out the perl.
    >
    > [Below is the command I use.]
    > sed -n "11353,/^$/p" filename.log


    sed '11353,$!d;/^$/q' filename.log

    would exit sed as soon as it reaches the empty line after the
    11353th.

    If you have several line numbers, you can do:

    sed '
    1000,/^$/b
    2000,/^$/b
    3000,/^$/b
    4000,$!d
    /^$/q' filename.log

    This will stop reading on the empty line after line 4000.

    Note that the line ranges must not overlap.

    --
    Stephane

+ Reply to Thread