deleting a penultimate line from the standard output - Unix

This is a discussion on deleting a penultimate line from the standard output - Unix ; I'm stuck with a simple problem: how to remove the penultimate line from the standard output by piping the output through sed? cat file | uniq | sed ??? '$-1d' etc. don't work... z.entropic...

+ Reply to Thread
Results 1 to 16 of 16

Thread: deleting a penultimate line from the standard output

  1. deleting a penultimate line from the standard output

    I'm stuck with a simple problem: how to remove the penultimate line
    from the standard output by piping the output through sed?

    cat file | uniq | sed ???

    '$-1d' etc. don't work...

    z.entropic


  2. Re: deleting a penultimate line from the standard output

    On 2006-02-15, z.entropic wrote:
    > I'm stuck with a simple problem: how to remove the penultimate line
    > from the standard output by piping the output through sed?
    >
    > cat file | uniq | sed ???


    Why cat?

    uniq file | ....

    > '$-1d' etc. don't work...


    awk 'NR < 3 { print }
    p2 != "" { print p2 }
    { p2 = p1; p1 = $0 }
    END { print }'

    --
    Chris F.A. Johnson, author |
    Shell Scripting Recipes: | My code in this post, if any,
    A Problem-Solution Approach | is released under the
    2005, Apress | GNU General Public Licence

  3. Re: deleting a penultimate line from the standard output

    On 15 Feb 2006 06:27:57 -0800, z.entropic wrote:
    > I'm stuck with a simple problem: how to remove the penultimate line
    > from the standard output by piping the output through sed?
    >
    > cat file | uniq | sed ???


    UUOC

    < file uniq | sed -e '$!N;${s/.*\n//;}' -e 'P;D'

    It won't remove anything if the file has less than 2 lines.

    $ seq 10 | sed -e '$!N;${s/.*\n//;}' -e 'P;D'
    1
    2
    3
    4
    5
    6
    7
    8
    10

    --
    Stephane

  4. Re: deleting a penultimate line from the standard output

    On Wed, 15 Feb 2006 09:38:38 -0500, Chris F.A. Johnson wrote:
    > On 2006-02-15, z.entropic wrote:
    >> I'm stuck with a simple problem: how to remove the penultimate line
    >> from the standard output by piping the output through sed?
    >>
    >> cat file | uniq | sed ???

    >
    > Why cat?
    >
    > uniq file | ....
    >
    >> '$-1d' etc. don't work...

    >
    > awk 'NR < 3 { print }
    > p2 != "" { print p2 }
    > { p2 = p1; p1 = $0 }
    > END { print }'


    $ seq 10 | awk 'NR < 3 { print }
    pipe quote> p2 != "" { print p2 }
    pipe quote> { p2 = p1; p1 = $0 }
    pipe quote> END { print }'
    1
    2
    1
    2
    3
    4
    5
    6
    7
    8

    $


    --
    Stephane

  5. Re: deleting a penultimate line from the standard output

    On 2006-02-15, Stephane Chazelas wrote:
    > On Wed, 15 Feb 2006 09:38:38 -0500, Chris F.A. Johnson wrote:
    >> On 2006-02-15, z.entropic wrote:
    >>> I'm stuck with a simple problem: how to remove the penultimate line
    >>> from the standard output by piping the output through sed?
    >>>
    >>> cat file | uniq | sed ???

    >>
    >> Why cat?
    >>
    >> uniq file | ....
    >>
    >>> '$-1d' etc. don't work...

    >>
    >> awk 'NR < 3 { print }
    >> p2 != "" { print p2 }
    >> { p2 = p1; p1 = $0 }
    >> END { print }'

    >
    > $ seq 10 | awk 'NR < 3 { print }
    > pipe quote> p2 != "" { print p2 }
    > pipe quote> { p2 = p1; p1 = $0 }
    > pipe quote> END { print }'
    > 1
    > 2
    > 1
    > 2
    > 3
    > 4
    > 5
    > 6
    > 7
    > 8
    >
    > $


    That's what you get for using the default awk on Sun/Solaris.

    I get it with old (pre 1988) awk; with nawk, gawk, mawk, and otawk
    I get:

    $ seq 10 | awk 'NR < 3 { print }
    > p2 != "" { print p2 }
    > { p2 = p1; p1 = $0 }
    > END { print }'

    1
    2
    1
    2
    3
    4
    5
    6
    7
    8
    10

    --
    Chris F.A. Johnson, author |
    Shell Scripting Recipes: | My code in this post, if any,
    A Problem-Solution Approach | is released under the
    2005, Apress | GNU General Public Licence

  6. Re: deleting a penultimate line from the standard output

    On Wed, 15 Feb 2006 10:19:36 -0500, Chris F.A. Johnson wrote:
    [...]
    > I get it with old (pre 1988) awk; with nawk, gawk, mawk, and otawk
    > I get:
    >
    > $ seq 10 | awk 'NR < 3 { print }
    >> p2 != "" { print p2 }
    >> { p2 = p1; p1 = $0 }
    >> END { print }'

    > 1
    > 2
    > 1
    > 2
    > 3
    > 4
    > 5
    > 6
    > 7
    > 8
    > 10


    On Solaris 7:

    $ seq 10 | nawk 'NR < 3 { print }
    p2 != "" { print p2 }
    { p2 = p1; p1 = $0 }
    END { print }'
    1
    2
    1
    2
    3
    4
    5
    6
    7
    8


    (it's OK with /usr/xpg4/bin/awk, though (except for see below))

    I don't think the OP asked for the first two lines to be
    repeated, BTW .

    --
    Stephane

  7. Re: deleting a penultimate line from the standard output

    On Wed, 15 Feb 2006 10:19:36 -0500, Chris F.A. Johnson wrote:
    [...]
    >>> awk 'NR < 3 { print }
    >>> p2 != "" { print p2 }
    >>> { p2 = p1; p1 = $0 }
    >>> END { print }'

    [...]
    > That's what you get for using the default awk on Sun/Solaris.
    >
    > I get it with old (pre 1988) awk; with nawk, gawk, mawk, and otawk
    > I get:

    [...]

    By the way, I can't find SUSv3 saying one can assume what the
    content of $0, $1... is in a BEGIN or END statement, but I've
    not looked hard. It does say that NF should retain the value for
    the last read record, though, so it would make sense that the
    $0, $1... from the last read record are retained as well.

    --
    Stephane

  8. Re: deleting a penultimate line from the standard output

    z.entropic wrote:
    > I'm stuck with a simple problem: how to remove the penultimate line
    > from the standard output by piping the output through sed?
    >
    > cat file | uniq | sed ???
    >
    > '$-1d' etc. don't work...
    >
    > z.entropic
    >


    If you have "tac":

    tac file | sed '2d' | tac

    Regards,

    Ed.

  9. Re: deleting a penultimate line from the standard output

    Stephane Chazelas wrote:
    > On 15 Feb 2006 06:27:57 -0800, z.entropic wrote:
    > > I'm stuck with a simple problem: how to remove the penultimate line
    > > from the standard output by piping the output through sed?
    > >
    > > cat file | uniq | sed ???

    >
    > UUOC
    >
    > < file uniq | sed -e '$!N;${s/.*\n//;}' -e 'P;D'
    >
    > It won't remove anything if the file has less than 2 lines.


    Wonderful!--but I don't understand the expression... Please translate
    (but please don't tell me to RTFM, although I do have it). ;-)

    z.entropic


  10. Re: deleting a penultimate line from the standard output

    2006-02-16, 10:14(-08), z.entropic:
    > Stephane Chazelas wrote:
    >> On 15 Feb 2006 06:27:57 -0800, z.entropic wrote:
    >> > I'm stuck with a simple problem: how to remove the penultimate line
    >> > from the standard output by piping the output through sed?
    >> >
    >> > cat file | uniq | sed ???

    >>
    >> UUOC
    >>
    >> < file uniq | sed -e '$!N;${s/.*\n//;}' -e 'P;D'
    >>
    >> It won't remove anything if the file has less than 2 lines.

    >
    > Wonderful!--but I don't understand the expression... Please translate
    > (but please don't tell me to RTFM, although I do have it). ;-)

    [...]

    Actually, this can be simply written:

    file uniq | sed -e '$!N;$s/.*\n//;P;D'

    sed normally can't tell it is on the penultimate line. It can
    tell if it's on the th line, or if it's on the last line.

    The idea is to read one line in advance, so that if we know that
    the next line is the last one, then we know the current one is
    the penultimate.

    $!N unless we are on the last line, append the next line to
    the pattern space (that increases the "current line
    number"). Unless there are less than 2 lines in the file,
    that will be the same as "N" alone. What that means is
    that for the rest of the commands, the pattern space will
    contain two consecutive lines.

    $s/.*\n//
    only if we are on the last line, remove the first of the
    two lines in the pattern space. Which means... the
    penultimate lines.

    P prints the first line of the pattern space (in case we are
    on the last line, that will be the last line)

    D remove the first line of the pattern space and loop from
    the beginning (unless the pattern space is empty as on the
    last line)

    so that we can append the next line to the pattern space
    and so on...

    --
    Stéphane

  11. Re: deleting a penultimate line from the standard output

    On 15 Feb 2006 06:27:57 -0800, "z.entropic" wrote:

    >I'm stuck with a simple problem: how to remove the penultimate line
    >from the standard output by piping the output through sed?
    >
    >cat file | uniq | sed ???
    >
    >'$-1d' etc. don't work...
    >
    >z.entropic
    >



    This post has already been answered plenty of times
    but here's another way;

    cat file | uniq | sed -n '$p;$q;x;1!p'

    $p $q -> on the last line print that line then quit
    so that we don't also print the previous line.

    x 1!p -> print the previous line for all lines except the
    first one.

    I wonder if it's possible to do this with only 3 commands
    or prove that it cannot be done with 3 commands otherwise?

    byefornow
    laura



    --
    echo alru_aafriehdab@ittnreen.tocm |sed 's/\(.\)\(.\)/\2\1/g'

  12. Re: deleting a penultimate line from the standard output

    On Thu, 2 Mar 2006 00:30:02 +0000 (UTC), run_signature_script_for_my_email@INVALID.com (laura fairhead) wrote:

    >On 15 Feb 2006 06:27:57 -0800, "z.entropic" wrote:
    >
    >>I'm stuck with a simple problem: how to remove the penultimate line
    >>from the standard output by piping the output through sed?
    >>
    >>cat file | uniq | sed ???
    >>
    >>'$-1d' etc. don't work...
    >>
    >>z.entropic
    >>

    >
    >
    >This post has already been answered plenty of times
    >but here's another way;
    >
    >cat file | uniq | sed -n '$p;$q;x;1!p'
    >
    >$p $q -> on the last line print that line then quit
    > so that we don't also print the previous line.
    >
    >x 1!p -> print the previous line for all lines except the
    > first one.
    >
    >I wonder if it's possible to do this with only 3 commands
    >or prove that it cannot be done with 3 commands otherwise?


    I was trying to prove it couldn't be done with only 3 commands
    and in the process found one solution;

    cat file | uniq | sed -e '$b' -e x -e 1d

    Still I wonder if you can prove it can't be possible in 2 commands;
    it surely can't !!?

    byefornow
    laura

    --
    echo alru_aafriehdab@ittnreen.tocm |sed 's/\(.\)\(.\)/\2\1/g'

  13. Re: deleting a penultimate line from the standard output

    laura fairhead wrote:
    > On Thu, 2 Mar 2006 00:30:02 +0000 (UTC), run_signature_script_for_my_email@INVALID.com (laura fairhead) wrote:
    >
    >> On 15 Feb 2006 06:27:57 -0800, "z.entropic" wrote:
    >>
    >>> I'm stuck with a simple problem: how to remove the penultimate line
    >> >from the standard output by piping the output through sed?
    >>> cat file | uniq | sed ???
    >>>
    >>> '$-1d' etc. don't work...
    >>>
    >>> z.entropic
    >>>

    >>
    >> This post has already been answered plenty of times
    >> but here's another way;
    >>
    >> cat file | uniq | sed -n '$p;$q;x;1!p'
    >>
    >> $p $q -> on the last line print that line then quit
    >> so that we don't also print the previous line.
    >>
    >> x 1!p -> print the previous line for all lines except the
    >> first one.
    >>
    >> I wonder if it's possible to do this with only 3 commands
    >> or prove that it cannot be done with 3 commands otherwise?

    >
    > I was trying to prove it couldn't be done with only 3 commands
    > and in the process found one solution;
    >
    > cat file | uniq | sed -e '$b' -e x -e 1d
    >


    I.e.

    uniq file | sed '$q;x;1d'


    --
    Michael Tosch @ hp : com

  14. Re: deleting a penultimate line from the standard output

    On Thu, 02 Mar 2006 12:52:22 +0100, Michael Tosch wrote:

    >laura fairhead wrote:
    >> On Thu, 2 Mar 2006 00:30:02 +0000 (UTC), run_signature_script_for_my_email@INVALID.com (laura fairhead) wrote:
    >>
    >>> On 15 Feb 2006 06:27:57 -0800, "z.entropic" wrote:
    >>>
    >>>> I'm stuck with a simple problem: how to remove the penultimate line
    >>> >from the standard output by piping the output through sed?
    >>>> cat file | uniq | sed ???
    >>>>
    >>>> '$-1d' etc. don't work...
    >>>>
    >>>> z.entropic
    >>>>
    >>>
    >>> This post has already been answered plenty of times
    >>> but here's another way;
    >>>
    >>> cat file | uniq | sed -n '$p;$q;x;1!p'
    >>>
    >>> $p $q -> on the last line print that line then quit
    >>> so that we don't also print the previous line.
    >>>
    >>> x 1!p -> print the previous line for all lines except the
    >>> first one.
    >>>
    >>> I wonder if it's possible to do this with only 3 commands
    >>> or prove that it cannot be done with 3 commands otherwise?

    >>
    >> I was trying to prove it couldn't be done with only 3 commands
    >> and in the process found one solution;
    >>
    >> cat file | uniq | sed -e '$b' -e x -e 1d
    >>

    >
    >I.e.
    >
    >uniq file | sed '$q;x;1d'
    >


    hI Michael,

    I suppose you mean by this shell commands ? I know I didn;t state
    it explicitly but I was referring to the number of 'sed' commands!
    (the first one had 4). Well, anyway, you've shown another possibilty
    with 3 commands I looked at proving you couldn't do it in 2
    and have got quite far into that proof (for example, split into
    two cases with and without '-n', without '-n' one of the lines needs
    to be deleted in most cases so that implies one of the 2 commands
    is a 'd' then you can show the other must be an 'x' and consequently
    how it must be impossible.... ).

    byefornow
    laura

    >
    >--
    >Michael Tosch @ hp : com


    --
    echo alru_aafriehdab@ittnreen.tocm |sed 's/\(.\)\(.\)/\2\1/g'

  15. Re: deleting a penultimate line from the standard output

    laura fairhead wrote:
    > On Thu, 02 Mar 2006 12:52:22 +0100, Michael Tosch wrote:
    >
    >> laura fairhead wrote:
    >>> On Thu, 2 Mar 2006 00:30:02 +0000 (UTC), run_signature_script_for_my_email@INVALID.com (laura fairhead) wrote:
    >>>
    >>>> On 15 Feb 2006 06:27:57 -0800, "z.entropic" wrote:
    >>>>
    >>>>> I'm stuck with a simple problem: how to remove the penultimate line
    >>>> >from the standard output by piping the output through sed?
    >>>>> cat file | uniq | sed ???
    >>>>>
    >>>>> '$-1d' etc. don't work...
    >>>>>
    >>>>> z.entropic
    >>>>>
    >>>> This post has already been answered plenty of times
    >>>> but here's another way;
    >>>>
    >>>> cat file | uniq | sed -n '$p;$q;x;1!p'
    >>>>
    >>>> $p $q -> on the last line print that line then quit
    >>>> so that we don't also print the previous line.
    >>>>
    >>>> x 1!p -> print the previous line for all lines except the
    >>>> first one.
    >>>>
    >>>> I wonder if it's possible to do this with only 3 commands
    >>>> or prove that it cannot be done with 3 commands otherwise?
    >>> I was trying to prove it couldn't be done with only 3 commands
    >>> and in the process found one solution;
    >>>
    >>> cat file | uniq | sed -e '$b' -e x -e 1d
    >>>

    >> I.e.
    >>
    >> uniq file | sed '$q;x;1d'
    >>

    >
    > hI Michael,
    >
    > I suppose you mean by this shell commands ? I know I didn;t state
    > it explicitly but I was referring to the number of 'sed' commands!
    > (the first one had 4). Well, anyway, you've shown another possibilty
    > with 3 commands I looked at proving you couldn't do it in 2
    > and have got quite far into that proof (for example, split into
    > two cases with and without '-n', without '-n' one of the lines needs
    > to be deleted in most cases so that implies one of the 2 commands
    > is a 'd' then you can show the other must be an 'x' and consequently
    > how it must be impossible.... ).
    >
    > byefornow
    > laura
    >


    I had no answer, and had cut off your question therefore.

    IMHO
    the first line and the last line must be treated differently.
    One common action, 2 exceptions => 3 commands at minimum.

    It is the minimum because sed's one hold buffer is perfect
    for it.

    How to implement sed '$-2d'? IMHO beyond sed's capabilities.

    P.S.
    echo Iil ky eo rusngiarute | sed 's/\(.\)\(.\)\(.\)\(.\)/\1\4\3\2/g'


    --
    Michael Tosch @ hp : com

  16. Re: deleting a penultimate line from the standard output

    On Thu, 02 Mar 2006 22:28:53 +0100, Michael Tosch wrote:

    >laura fairhead wrote:
    >> On Thu, 02 Mar 2006 12:52:22 +0100, Michael Tosch wrote:
    >>
    >>> laura fairhead wrote:
    >>>> On Thu, 2 Mar 2006 00:30:02 +0000 (UTC), run_signature_script_for_my_email@INVALID.com (laura fairhead) wrote:
    >>>>
    >>>>> On 15 Feb 2006 06:27:57 -0800, "z.entropic" wrote:
    >>>>>
    >>>>>> I'm stuck with a simple problem: how to remove the penultimate line
    >>>>> >from the standard output by piping the output through sed?
    >>>>>> cat file | uniq | sed ???
    >>>>>>
    >>>>>> '$-1d' etc. don't work...
    >>>>>>
    >>>>>> z.entropic
    >>>>>>
    >>>>> This post has already been answered plenty of times
    >>>>> but here's another way;
    >>>>>
    >>>>> cat file | uniq | sed -n '$p;$q;x;1!p'
    >>>>>
    >>>>> $p $q -> on the last line print that line then quit
    >>>>> so that we don't also print the previous line.
    >>>>>
    >>>>> x 1!p -> print the previous line for all lines except the
    >>>>> first one.
    >>>>>
    >>>>> I wonder if it's possible to do this with only 3 commands
    >>>>> or prove that it cannot be done with 3 commands otherwise?
    >>>> I was trying to prove it couldn't be done with only 3 commands
    >>>> and in the process found one solution;
    >>>>
    >>>> cat file | uniq | sed -e '$b' -e x -e 1d
    >>>>
    >>> I.e.
    >>>
    >>> uniq file | sed '$q;x;1d'
    >>>

    >>
    >> hI Michael,
    >>
    >> I suppose you mean by this shell commands ? I know I didn;t state
    >> it explicitly but I was referring to the number of 'sed' commands!
    >> (the first one had 4). Well, anyway, you've shown another possibilty
    >> with 3 commands I looked at proving you couldn't do it in 2
    >> and have got quite far into that proof (for example, split into
    >> two cases with and without '-n', without '-n' one of the lines needs
    >> to be deleted in most cases so that implies one of the 2 commands
    >> is a 'd' then you can show the other must be an 'x' and consequently
    >> how it must be impossible.... ).
    >>
    >> byefornow
    >> laura
    >>

    >
    >I had no answer, and had cut off your question therefore.
    >
    >IMHO
    >the first line and the last line must be treated differently.
    >One common action, 2 exceptions => 3 commands at minimum.
    >
    >It is the minimum because sed's one hold buffer is perfect
    >for it.
    >
    >How to implement sed '$-2d'? IMHO beyond sed's capabilities.


    sed -n '1!x;3,${$!P;s/.*\n//;};1!G;$p;h'

    Since you are operating on a stream you will either use a program
    in 'sed' or 'awk' with a window buffer mechanism code. Otherwise
    one could deserialise the stream (writing it to a temporary file)
    and use 'ed' to delete the line;

    data_produce_program >f
    ed f

    >
    >P.S.
    >echo Iil ky eo rusngiarute | sed 's/\(.\)\(.\)\(.\)\(.\)/\1\4\3\2/g'
    >




    bestwishes
    laura

    >
    >--
    >Michael Tosch @ hp : com


    --
    echo alru_aafriehdab@ittnreen.tocm |sed 's/\(.\)\(.\)/\2\1/g'

+ Reply to Thread