grep for SOH char (\x01) - Unix

This is a discussion on grep for SOH char (\x01) - Unix ; I am trying to grep for the chat hex 0x01 the SOH char. This is for FIX protocol (a financial protocol). The lines are like this: 35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000 35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000 35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000 where ^A is the hex value 0x01 When I try this ...

+ Reply to Thread
Results 1 to 5 of 5

Thread: grep for SOH char (\x01)

  1. grep for SOH char (\x01)

    I am trying to grep for the chat hex 0x01 the SOH char. This is for
    FIX protocol (a financial protocol). The lines are like this:

    35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000
    35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000
    35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000


    where ^A is the hex value 0x01

    When I try this on input file:

    grep -E '35=D'

    I get the matches that I am expecting, but none of this match any
    lines:

    grep -E '\x0135=D\x01'
    grep -E '35=D\x01'

    Not sure why? Perhaps someone can help.

    I need to use the SOH delimiter because '35=D' is too generic. It
    should only count when it is between SOH chars. Here is the version of
    grep and linux:

    grep (GNU grep) 2.5.1

    Linux devlot13012 2.6.18-8.1.1.el5 #1 SMP Mon Feb 26 20:37:57 EST 2007
    x86_64 x86_64 x86_64 GNU/Linux

  2. Re: grep for SOH char (\x01)

    2008-04-25, 13:17(-07), Orson:
    > I am trying to grep for the chat hex 0x01 the SOH char. This is for
    > FIX protocol (a financial protocol). The lines are like this:
    >
    > 35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000
    > 35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000
    > 35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000
    >
    >
    > where ^A is the hex value 0x01
    >
    > When I try this on input file:
    >
    > grep -E '35=D'
    >
    > I get the matches that I am expecting, but none of this match any
    > lines:
    >
    > grep -E '\x0135=D\x01'
    > grep -E '35=D\x01'
    >
    > Not sure why? Perhaps someone can help.


    grep '\x01' matches "x01".

    With some shells, you can do

    grep $'\x01'

    or

    grep $'\01'

    Standardly, you can do:

    SOH=$(printf '\1')
    grep "$SOH"

    Or

    grep '^A'

    where ^A is the SOH character ( in vi).

    You could also use awk:

    awk '/\1/'

    --
    Stéphane

  3. Re: grep for SOH char (\x01)

    On Apr 26, 1:34*pm, Stephane CHAZELAS wrote:
    > 2008-04-25, 13:17(-07), Orson:
    >
    >
    >
    >
    >
    > > I am trying to grep for the chat hex 0x01 the SOH char. This is for
    > > FIX protocol (a financial protocol). The lines are like this:

    >
    > > 35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000
    > > 35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000
    > > 35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000

    >
    > > where ^A is the hex value 0x01

    >
    > > When I try this on input file:

    >
    > > grep -E '35=D'

    >
    > > I get the matches that I am expecting, but none of this match any
    > > lines:

    >
    > > grep -E '\x0135=D\x01'
    > > grep -E '35=D\x01'

    >
    > > Not sure why? Perhaps someone can help.

    >
    > grep '\x01' matches "x01".
    >

    Well, grep '\x01' does not return any lines for me given the input
    above. Does it work for you? I am running:
    grep (GNU grep) 2.5.1

    uname -a
    is:
    Linux devlot13012 2.6.18-8.1.1.el5 #1 SMP Mon Feb 26 20:37:57 EST
    2007
    x86_64 x86_64 x86_64 GNU/Linux


  4. Re: grep for SOH char (\x01)

    2008-04-29, 06:44(-07), Orson:
    > On Apr 26, 1:34*pm, Stephane CHAZELAS wrote:

    [...]
    >> > 35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000
    >> > 35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000
    >> > 35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000

    [...]
    >> grep '\x01' matches "x01".
    >>

    > Well, grep '\x01' does not return any lines for me given the input
    > above. Does it work for you?


    Well, I can't see any "x01" in that input.

    $ echo box01 | grep '\x01'
    box01

    works for me.

    So does:

    $ printf 'xx\1yy\n' | awk '/\1/' | od -c
    0000000 x x 001 y y \n
    0000006


    --
    Stéphane

  5. Re: grep for SOH char (\x01)

    On Apr 29, 11:07*am, Stephane CHAZELAS
    wrote:
    > 2008-04-29, 06:44(-07), Orson:
    >
    > > On Apr 26, 1:34*pm, Stephane CHAZELAS wrote:

    > [...]
    > >> > 35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000
    > >> > 35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000
    > >> > 35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000

    > [...]
    > >> grep '\x01' matches "x01".

    >
    > > Well, grep '\x01' does not return any lines for me given the input
    > > above. Does it work for you?

    >
    > Well, I can't see any "x01" in that input.
    >
    > $ echo box01 | grep '\x01'
    > box01
    >
    > works for me.
    >
    > So does:
    >
    > $ printf 'xx\1yy\n' | awk '/\1/' | od -c
    > 0000000 * x * x 001 * y * y *\n
    > 0000006
    >
    > --
    > Stéphane

    A solution for grep is:

    grep -P "(35=D\x01)|(35=E)\x01"

    The -P will use perl regular expressions and this works. As said in my
    original post, the input provided has ^A in place of \x01 (remember
    that \x01 is non-printable ascii).

    Also, the command:

    echo box01 does not produce the SOH char (hex value 0x01). If you pipe
    this to file and then open with a hex editor you will see it has the
    characters 'b' 'o' 'x' '0' '1'. To produce my test file, I created
    java program to output the non-printable ascii character SOH (hex
    value 0x01).



+ Reply to Thread