grep doesn't know '\xHH' ? - Unix

This is a discussion on grep doesn't know '\xHH' ? - Unix ; Please see below: $ echo -e '\xde\xad\xbe\xef' > test.txt $ od -t x1 test.txt 0000000 de ad be ef 0a 0000005 $ grep -b -E '\xef' test.txt $ echo $? 1 $ grep --version grep (GNU grep) 2.5.1 I am ...

+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 21

Thread: grep doesn't know '\xHH' ?

  1. grep doesn't know '\xHH' ?


    Please see below:

    $ echo -e '\xde\xad\xbe\xef' > test.txt
    $ od -t x1 test.txt
    0000000 de ad be ef 0a
    0000005
    $ grep -b -E '\xef' test.txt
    $ echo $?
    1
    $ grep --version
    grep (GNU grep) 2.5.1


    I am using Linux and GNU grep. The grep doesn't know '\xHH' pattern?
    Or I miss something obvious? :-)

    Thanks!


  2. Re: grep doesn't know '\xHH' ?

    On Aug 22, 9:59*am, WANG Cong wrote:
    > Please see below:
    >
    > $ echo -e '\xde\xad\xbe\xef' > test.txt
    > $ od -t x1 test.txt
    > 0000000 de ad be ef 0a
    > 0000005
    > $ grep -b -E '\xef' test.txt
    > $ echo $?
    > 1
    > $ grep --version
    > grep (GNU grep) 2.5.1
    >
    > I am using Linux and GNU grep. The grep doesn't know '\xHH' pattern?
    > Or I miss something obvious? :-)
    >
    > Thanks!


    man echo

    On my platforms, \x is NOT one of the escaped characters recognized
    by the echo command.
    --
    Fred Kleinschmidt

  3. Re: grep doesn't know '\xHH' ?

    On 22 Ago, 20:20, Fred wrote:
    > On Aug 22, 9:59*am, WANG Cong wrote:
    >
    >
    >
    > > Please see below:

    >
    > > $ echo -e '\xde\xad\xbe\xef' > test.txt
    > > $ od -t x1 test.txt
    > > 0000000 de ad be ef 0a
    > > 0000005
    > > $ grep -b -E '\xef' test.txt
    > > $ echo $?
    > > 1
    > > $ grep --version
    > > grep (GNU grep) 2.5.1

    >
    > > I am using Linux and GNU grep. The grep doesn't know '\xHH' pattern?
    > > Or I miss something obvious? :-)

    >
    > > Thanks!

    >
    > man echo
    >
    > On my platforms, \x is NOT one of the escaped characters recognized
    > by the echo command.
    > --
    > Fred Kleinschmidt


    You're right, but echo's often overridden by the shell builtin one
    (e.g. bash's echo support that). So, first of all, he should take a
    look to: % type echo

    s.

  4. Re: grep doesn't know '\xHH' ?

    WANG Cong wrote:
    > Please see below:
    >
    > $ echo -e '\xde\xad\xbe\xef' > test.txt
    > $ od -t x1 test.txt
    > 0000000 de ad be ef 0a
    > 0000005
    > $ grep -b -E '\xef' test.txt
    > $ echo $?
    > 1
    > $ grep --version
    > grep (GNU grep) 2.5.1
    >
    >
    > I am using Linux and GNU grep. The grep doesn't know '\xHH' pattern?
    > Or I miss something obvious? :-)
    >
    > Thanks!
    >


    No. grep doesn't know '\xHH' patterns; it knows regular expressions.

    Robert

  5. Re: grep doesn't know '\xHH' ?

    Sergio Perticone writes:

    > On 22 Ago, 20:20, Fred wrote:
    >> On Aug 22, 9:59*am, WANG Cong wrote:
    >>
    >>
    >>
    >> > Please see below:

    >>
    >> > $ echo -e '\xde\xad\xbe\xef' > test.txt
    >> > $ od -t x1 test.txt
    >> > 0000000 de ad be ef 0a
    >> > 0000005
    >> > $ grep -b -E '\xef' test.txt
    >> > $ echo $?
    >> > 1
    >> > $ grep --version
    >> > grep (GNU grep) 2.5.1

    >>
    >> > I am using Linux and GNU grep. The grep doesn't know '\xHH' pattern?
    >> > Or I miss something obvious? :-)


    Depends on what you think is obvious. grep will do its stuff if you
    cat give it the character. Because I have a UTF-8 terminal, to test I
    used bytes that form a valid sequence (right arrow) but that is not
    important:

    $ echo -e 'abc\ndef\xe2\x86\x92\nghi' > test.txt
    $ cat test.txt
    abc
    def→
    ghi
    od -t x1 test.txt
    0000000 61 62 63 0a 64 65 66 e2 86 92 0a 67 68 69 0a
    0000017

    Now for grep:

    $ grep '\xe2' test.txt

    No. grep does not translate \x sequences. But bash has a tricks up
    its sleeves:

    $ grep $'\xe2' test.txt
    def→
    $ grep `echo -e '\xe2'` test.txt
    def→
    $ grep $(echo -e '\xe2') test.txt
    def→

    You have other trouble if the file in no line-oriented. grep knows
    only lines.

    --
    Ben.

  6. Re: grep doesn't know '\xHH' ?

    Ben Bacarisse writes:

    > Sergio Perticone writes:


    Sorry, my reply was intended to be to the original messsage. I was
    not commenting you what you wrote. I hope the OP sees that it is
    intended for them!

    --
    Ben.

  7. Re: grep doesn't know '\xHH' ?

    On 822, 10ʱ33, Robert Harris
    wrote:
    > WANG Cong wrote:
    > > Please see below:

    >
    > > $ echo -e '\xde\xad\xbe\xef' > test.txt
    > > $ od -t x1 test.txt
    > > 0000000 de ad be ef 0a
    > > 0000005
    > > $ grep -b -E '\xef' test.txt
    > > $ echo $?
    > > 1
    > > $ grep --version
    > > grep (GNU grep) 2.5.1

    >
    > > I am using Linux and GNU grep. The grep doesn't know '\xHH' pattern?
    > > Or I miss something obvious? :-)

    >
    > > Thanks!

    >
    > No. grep doesn't know '\xHH' patterns; it knows regular expressions.


    Thanks. But according to this[1], '\xHH' is a kind of regular
    expressions.

    http://www.araxis.com/merge/topic_regexpreference.html

  8. Re: grep doesn't know '\xHH' ?

    On 823, 12ʱ17, Ben Bacarisse wrote:
    > Sergio Perticone writes:
    > > On 22 Ago, 20:20, Fred wrote:
    > >> On Aug 22, 9:59 am, WANG Cong wrote:

    >
    > >> > Please see below:

    >
    > >> > $ echo -e '\xde\xad\xbe\xef' > test.txt
    > >> > $ od -t x1 test.txt
    > >> > 0000000 de ad be ef 0a
    > >> > 0000005
    > >> > $ grep -b -E '\xef' test.txt
    > >> > $ echo $?
    > >> > 1
    > >> > $ grep --version
    > >> > grep (GNU grep) 2.5.1

    >
    > >> > I am using Linux and GNU grep. The grep doesn't know '\xHH' pattern?
    > >> > Or I miss something obvious? :-)

    >
    > Depends on what you think is obvious. grep will do its stuff if you
    > cat give it the character. Because I have a UTF-8 terminal, to test I
    > used bytes that form a valid sequence (right arrow) but that is not
    > important:
    >
    > $ echo -e 'abc\ndef\xe2\x86\x92\nghi' > test.txt
    > $ cat test.txt
    > abc
    > def
    > ghi
    > od -t x1 test.txt
    > 0000000 61 62 63 0a 64 65 66 e2 86 92 0a 67 68 69 0a
    > 0000017
    >
    > Now for grep:
    >
    > $ grep '\xe2' test.txt
    >
    > No. grep does not translate \x sequences. But bash has a tricks up
    > its sleeves:
    >
    > $ grep $'\xe2' test.txt
    > def
    > $ grep `echo -e '\xe2'` test.txt
    > def
    > $ grep $(echo -e '\xe2') test.txt
    > def
    >
    > You have other trouble if the file in no line-oriented. grep knows
    > only lines.


    Thanks for your detailed explanation. If so, then how can I search
    a \x sequence in a binary file? Of course, I mean using the existing
    shell command, and I want to get the offset of a given \x sequence
    in a binary file.


  9. Re: grep doesn't know '\xHH' ?

    WANG Cong writes:

    > On 8月23日, 上午12时17分, Ben Bacarisse wrote:
    >> Sergio Perticone writes:
    >> > On 22 Ago, 20:20, Fred wrote:
    >> >> On Aug 22, 9:59 am, WANG Cong wrote:

    >>
    >> >> > Please see below:

    >>
    >> >> > $ echo -e '\xde\xad\xbe\xef' > test.txt
    >> >> > $ od -t x1 test.txt
    >> >> > 0000000 de ad be ef 0a
    >> >> > 0000005
    >> >> > $ grep -b -E '\xef' test.txt
    >> >> > $ echo $?
    >> >> > 1
    >> >> > $ grep --version
    >> >> > grep (GNU grep) 2.5.1

    >>
    >> >> > I am using Linux and GNU grep. The grep doesn't know '\xHH' pattern?
    >> >> > Or I miss something obvious? :-)

    >>
    >> Depends on what you think is obvious. grep will do its stuff if you
    >> cat give it the character. Because I have a UTF-8 terminal, to test I
    >> used bytes that form a valid sequence (right arrow) but that is not
    >> important:
    >>
    >> $ echo -e 'abc\ndef\xe2\x86\x92\nghi' > test.txt
    >> $ cat test.txt
    >> abc
    >> def→
    >> ghi
    >> od -t x1 test.txt
    >> 0000000 61 62 63 0a 64 65 66 e2 86 92 0a 67 68 69 0a
    >> 0000017
    >>
    >> Now for grep:
    >>
    >> $ grep '\xe2' test.txt
    >>
    >> No. grep does not translate \x sequences. But bash has a tricks up
    >> its sleeves:
    >>
    >> $ grep $'\xe2' test.txt
    >> def→
    >> $ grep `echo -e '\xe2'` test.txt
    >> def→
    >> $ grep $(echo -e '\xe2') test.txt
    >> def→
    >>
    >> You have other trouble if the file in no line-oriented. grep knows
    >> only lines.

    >
    > Thanks for your detailed explanation. If so, then how can I search
    > a \x sequence in a binary file? Of course, I mean using the existing
    > shell command, and I want to get the offset of a given \x sequence
    > in a binary file.


    grep --binary-files=text --byte-offset --only-matching $'\xaa\xbb'

    or with short flags:

    grep -abo $'\xaa\xbb'

    gets close and

    grep -abo $'\xaa\xbb' | cut -d: -f1
    or
    grep -abo $'\xaa\xbb' | cut -d: -f1 | head -1

    is probably exactly what you want. Do you have a manual for the system
    you are using? All the flags for grep should be documented by "man grep".

    --
    Ben.

  10. Re: grep doesn't know '\xHH' ?

    On 824, 4ʱ17, Ben Bacarisse wrote:
    > WANG Cong writes:
    > > On 823, 12ʱ17, Ben Bacarisse wrote:
    > >> Sergio Perticone writes:
    > >> > On 22 Ago, 20:20, Fred wrote:
    > >> >> On Aug 22, 9:59 am, WANG Cong wrote:

    >
    > >> >> > Please see below:

    >
    > >> >> > $ echo -e '\xde\xad\xbe\xef' > test.txt
    > >> >> > $ od -t x1 test.txt
    > >> >> > 0000000 de ad be ef 0a
    > >> >> > 0000005
    > >> >> > $ grep -b -E '\xef' test.txt
    > >> >> > $ echo $?
    > >> >> > 1
    > >> >> > $ grep --version
    > >> >> > grep (GNU grep) 2.5.1

    >
    > >> >> > I am using Linux and GNU grep. The grep doesn't know '\xHH' pattern?
    > >> >> > Or I miss something obvious? :-)

    >
    > >> Depends on what you think is obvious. grep will do its stuff if you
    > >> cat give it the character. Because I have a UTF-8 terminal, to test I
    > >> used bytes that form a valid sequence (right arrow) but that is not
    > >> important:

    >
    > >> $ echo -e 'abc\ndef\xe2\x86\x92\nghi' > test.txt
    > >> $ cat test.txt
    > >> abc
    > >> def
    > >> ghi
    > >> od -t x1 test.txt
    > >> 0000000 61 62 63 0a 64 65 66 e2 86 92 0a 67 68 69 0a
    > >> 0000017

    >
    > >> Now for grep:

    >
    > >> $ grep '\xe2' test.txt

    >
    > >> No. grep does not translate \x sequences. But bash has a tricks up
    > >> its sleeves:

    >
    > >> $ grep $'\xe2' test.txt
    > >> def
    > >> $ grep `echo -e '\xe2'` test.txt
    > >> def
    > >> $ grep $(echo -e '\xe2') test.txt
    > >> def

    >
    > >> You have other trouble if the file in no line-oriented. grep knows
    > >> only lines.

    >
    > > Thanks for your detailed explanation. If so, then how can I search
    > > a \x sequence in a binary file? Of course, I mean using the existing
    > > shell command, and I want to get the offset of a given \x sequence
    > > in a binary file.

    >
    > grep --binary-files=text --byte-offset --only-matching $'\xaa\xbb'
    >
    > or with short flags:
    >
    > grep -abo $'\xaa\xbb'
    >
    > gets close and
    >
    > grep -abo $'\xaa\xbb' | cut -d: -f1
    > or
    > grep -abo $'\xaa\xbb' | cut -d: -f1 | head -1
    >
    > is probably exactly what you want. Do you have a manual for the system
    > you are using? All the flags for grep should be documented by "man grep"..
    >


    I surely tried this, but it doesn't work well. see below:

    $ echo -e '\xde\xad\xbe\xef' > test.bin
    $ grep -abo $'\xef' test.bin
    0:

    That is wrong, it should output 3. :-(

  11. Re: grep doesn't know '\xHH' ?

    WANG Cong writes:

    > On 8月24日, 下午4时17分, Ben Bacarisse wrote:


    >>
    >> grep --binary-files=text --byte-offset --only-matching $'\xaa\xbb'
    >>
    >> or with short flags:
    >>
    >> grep -abo $'\xaa\xbb'
    >>
    >> gets close and
    >>
    >> grep -abo $'\xaa\xbb' | cut -d: -f1
    >> or
    >> grep -abo $'\xaa\xbb' | cut -d: -f1 | head -1
    >>
    >> is probably exactly what you want. Do you have a manual for the system
    >> you are using? All the flags for grep should be documented by "man grep".
    >>

    >
    > I surely tried this, but it doesn't work well. see below:
    >
    > $ echo -e '\xde\xad\xbe\xef' > test.bin
    > $ grep -abo $'\xef' test.bin
    > 0:
    >
    > That is wrong, it should output 3. :-(


    I get 3.

    --
    Ben.

  12. Re: grep doesn't know '\xHH' ?

    On Aug 24, 10:35 am, WANG Cong wrote:
    > On 824, 4ʱ17, Ben Bacarisse wrote:
    >
    >
    >
    > > WANG Cong writes:
    > > > On 823, 12ʱ17, Ben Bacarisse wrote:

    ...
    > > >> No. grep does not translate \x sequences. But bash has a
    > > >> tricks up its sleeves:

    >
    > > >> $ grep $'\xe2' test.txt
    > > >> def
    > > >> $ grep `echo -e '\xe2'` test.txt
    > > >> def

    ...
    > I surely tried this, but it doesn't work well. see below:
    >
    > $ echo -e '\xde\xad\xbe\xef' > test.bin
    > $ grep -abo $'\xef' test.bin
    > 0:
    >
    > That is wrong, it should output 3. :-(


    As Ben suggested in his comment that mentioned the $'...' trick,
    that's a feature of bash. Have you confirmed that you're running a
    version of bash that's new enough to support that feature? What's the
    output of
    printf %s $'\xef' | od -h

    ? It should be
    0000000 00fe
    0000001

    If that works fine, then use 'od' to check the contents of test.bin
    after that 'echo' and see if it's what you expect.


    Philip Guenther

  13. Re: grep doesn't know '\xHH' ?

    On Aug 25, 4:02, "guent...@gmail.com" wrote:
    > On Aug 24, 10:35 am, WANG Cong wrote:
    >
    >
    >
    > > On 8月24日, 下午4时17分, Ben Bacarisse wrote:

    >
    > > > WANG Cong writes:
    > > > > On 8月23日, 上午12时17分,Ben Bacarisse wrote:

    > ...
    > > > >> No. grep does not translate \x sequences. *But bash has a
    > > > >> tricks up its sleeves:

    >
    > > > >> $ grep $'\xe2' test.txt
    > > > >> def→
    > > > >> $ grep `echo -e '\xe2'` test.txt
    > > > >> def→

    > ...
    > > I surely tried this, but it doesn't work well. see below:

    >
    > > $ echo -e '\xde\xad\xbe\xef' > test.bin
    > > $ grep -abo $'\xef' *test.bin
    > > 0:

    >
    > > That is wrong, it should output 3. *:-(

    >
    > As Ben suggested in his comment that mentioned the $'...' trick,
    > that's a feature of bash. *Have you confirmed that you're running a
    > version of bash that's new enough to support that feature? *What's the
    > output of
    > * * printf %s $'\xef' | od -h
    >
    > ? *It should be
    > * * 0000000 00fe
    > * * 0000001
    >



    $ printf %s $'\xef' | od -h
    0000000 00ef
    0000001
    $ bash --version
    GNU bash, version 3.2.33(1)-release (i386-redhat-linux-gnu)
    Copyright (C) 2007 Free Software Foundation, Inc.

    I think it's new enough, right?

    > If that works fine, then use 'od' to check the contents of test.bin
    > after that 'echo' and see if it's what you expect.


    Yes,

    $ echo -e '\xde\xad\xbe\xef' > test.txt
    $ od -t x1 test.txt
    0000000 de ad be ef 0a
    0000005
    $ grep -abo $'\xef' test.txt
    0:�

    Strange, since Ben can get the correct result...

    Thanks, everyone!


  14. Re: grep doesn't know '\xHH' ?

    WANG Cong writes:

    > $ echo -e '\xde\xad\xbe\xef' > test.txt
    > $ od -t x1 test.txt
    > 0000000 de ad be ef 0a
    > 0000005
    > $ grep -abo $'\xef' test.txt
    > 0:�
    >
    > Strange, since Ben can get the correct result...


    A couple of things:

    $ grep --version
    GNU grep 2.5.3

    Maybe your grep is slightly different. Also check all the environment
    variables that alter grep's behaviour so try:

    $ env -i grep -abo $'\xef' test.txt

    --
    Ben.

  15. Re: grep doesn't know '\xHH' ?

    On Aug 25, 1:08, Ben Bacarisse wrote:
    > WANG Cong writes:
    >
    >
    >
    > > $ echo -e '\xde\xad\xbe\xef' > test.txt
    > > $ od -t x1 test.txt
    > > 0000000 de ad be ef 0a
    > > 0000005
    > > $ grep -abo $'\xef' *test.txt
    > > 0:

    >
    > > Strange, since Ben can get the correct result...

    >
    > A couple of things:
    >
    > $ grep --version
    > GNU grep 2.5.3


    Mine is a little different:

    $ grep --version
    grep (GNU grep) 2.5.1


    >
    > Maybe your grep is slightly different. *Also check all the environment
    > variables that alter grep's behaviour so try:
    >
    > $ env -i grep -abo $'\xef' test.txt
    >


    Still the same result.


  16. Re: grep doesn't know '\xHH' ?

    WANG Cong writes:

    > On Aug 25, 1:08, Ben Bacarisse wrote:
    >> WANG Cong writes:
    >>
    >>
    >>
    >> > $ echo -e '\xde\xad\xbe\xef' > test.txt
    >> > $ od -t x1 test.txt
    >> > 0000000 de ad be ef 0a
    >> > 0000005
    >> > $ grep -abo $'\xef' *test.txt
    >> > 0:

    >>
    >> > Strange, since Ben can get the correct result...

    >>
    >> A couple of things:
    >>
    >> $ grep --version
    >> GNU grep 2.5.3

    >
    > Mine is a little different:
    >
    > $ grep --version
    > grep (GNU grep) 2.5.1


    The bug tracker reports that there is an bug with --byte-offset (-b)
    in that version.

    --
    Ben.

  17. Re: grep doesn't know '\xHH' ?

    On Aug 25, 10:49 pm, Ben Bacarisse wrote:
    > WANG Cong writes:
    > > On Aug 25, 1:08, Ben Bacarisse wrote:
    > >> WANG Cong writes:

    >
    > >>

    >
    > >> > $ echo -e '\xde\xad\xbe\xef' > test.txt
    > >> > $ od -t x1 test.txt
    > >> > 0000000 de ad be ef 0a
    > >> > 0000005
    > >> > $ grep -abo $'\xef' *test.txt
    > >> > 0:

    >
    > >> > Strange, since Ben can get the correct result...

    >
    > >> A couple of things:

    >
    > >> $ grep --version
    > >> GNU grep 2.5.3

    >
    > > Mine is a little different:

    >
    > > $ grep --version
    > > grep (GNU grep) 2.5.1

    >
    > The bug tracker reports that there is an bug with --byte-offset (-b)
    > in that version.


    I see, so I will update my grep!

    Thanks, Ben.


  18. Re: grep doesn't know '\xHH' ?


    On Aug 24 2008 20:02, guenther@gmail.com wrote:
    > ...
    >>>>> No. grep does not translate \x sequences. But bash has a
    >>>>> tricks up its sleeves:

    >>
    >>>>> $ grep $'\xe2' test.txt
    >>>>> def



    grep -P "\xe2" test.txt

  19. Re: grep doesn't know '\xHH' ?

    Jan Engelhardt writes:

    > On Aug 24 2008 20:02, guenther@gmail.com wrote:
    >> ...
    >>>>>> No. grep does not translate \x sequences. But bash has a
    >>>>>> tricks up its sleeves:
    >>>
    >>>>>> $ grep $'\xe2' test.txt
    >>>>>> def→

    >
    > grep -P "\xe2" test.txt


    I was going to suggest that until I found out that on my system (and
    thus probably on lots of others) grep is not built with Perl regexps.

    --
    Ben.

  20. Re: grep doesn't know '\xHH' ?


    On Sep 24 2008 22:16, Ben Bacarisse wrote:

    >>> No. grep does not translate \x sequences. But bash has a
    >>> tricks up its sleeves:
    >>> $ grep $'\xe2' test.txt
    >>> def→

    >>
    >> grep -P "\xe2" test.txt

    >
    > I was going to suggest that until I found out that on my system (and
    > thus probably on lots of others) grep is not built with Perl regexps.


    Complain to your distro then, because mine has it and it's really a
    life saver :-)

+ Reply to Thread
Page 1 of 2 1 2 LastLast