grep for SOH char (\x01)

I am trying to grep for the chat hex 0x01 the SOH char. This is for
FIX protocol (a financial protocol). The lines are like this:

35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000
35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000
35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000


where ^A is the hex value 0x01

When I try this on input file:

grep -E '35=D'

I get the matches that I am expecting, but none of this match any
lines:

grep -E '\x0135=D\x01'
grep -E '35=D\x01'

Not sure why? Perhaps someone can help.

I need to use the SOH delimiter because '35=D' is too generic. It
should only count when it is between SOH chars. Here is the version of
grep and linux:

grep (GNU grep) 2.5.1

Linux devlot13012 2.6.18-8.1.1.el5 #1 SMP Mon Feb 26 20:37:57 EST 2007
x86_64 x86_64 x86_64 GNU/Linux

2008-04-25, 13:17(-07), Orson:[color=blue]
> I am trying to grep for the chat hex 0x01 the SOH char. This is for
> FIX protocol (a financial protocol). The lines are like this:
>
> 35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000
> 35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000
> 35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000
>
>
> where ^A is the hex value 0x01
>
> When I try this on input file:
>
> grep -E '35=D'
>
> I get the matches that I am expecting, but none of this match any
> lines:
>
> grep -E '\x0135=D\x01'
> grep -E '35=D\x01'
>
> Not sure why? Perhaps someone can help.[/color]

grep '\x01' matches "x01".

With some shells, you can do

grep $'\x01'

or

grep $'\01'

Standardly, you can do:

SOH=$(printf '\1')
grep "$SOH"

Or

grep '^A'

where ^A is the SOH character (<Ctrl-V><Ctrl-A> in vi).

You could also use awk:

awk '/\1/'

--
Stéphane

On Apr 26, 1:34*pm, Stephane CHAZELAS <this.addr...@is.invalid> wrote:[color=blue]
> 2008-04-25, 13:17(-07), Orson:
>
>
>
>
>[color=green]
> > I am trying to grep for the chat hex 0x01 the SOH char. This is for
> > FIX protocol (a financial protocol). The lines are like this:[/color]
>[color=green]
> > 35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000
> > 35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000
> > 35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000[/color]
>[color=green]
> > where ^A is the hex value 0x01[/color]
>[color=green]
> > When I try this on input file:[/color]
>[color=green]
> > grep -E '35=D'[/color]
>[color=green]
> > I get the matches that I am expecting, but none of this match any
> > lines:[/color]
>[color=green]
> > grep -E '\x0135=D\x01'
> > grep -E '35=D\x01'[/color]
>[color=green]
> > Not sure why? Perhaps someone can help.[/color]
>
> grep '\x01' matches "x01".
>[/color]
Well, grep '\x01' does not return any lines for me given the input
above. Does it work for you? I am running:
grep (GNU grep) 2.5.1

uname -a
is:
Linux devlot13012 2.6.18-8.1.1.el5 #1 SMP Mon Feb 26 20:37:57 EST
2007
x86_64 x86_64 x86_64 GNU/Linux

2008-04-29, 06:44(-07), Orson:[color=blue]
> On Apr 26, 1:34*pm, Stephane CHAZELAS <this.addr...@is.invalid> wrote:[/color]
[...][color=blue][color=green][color=darkred]
>> > 35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000
>> > 35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000
>> > 35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000[/color][/color][/color]
[...][color=blue][color=green]
>> grep '\x01' matches "x01".
>>[/color]
> Well, grep '\x01' does not return any lines for me given the input
> above. Does it work for you?[/color]

Well, I can't see any "x01" in that input.

$ echo box01 | grep '\x01'
box01

works for me.

So does:

$ printf 'xx\1yy\n' | awk '/\1/' | od -c
0000000 x x 001 y y \n
0000006


--
Stéphane

On Apr 29, 11:07*am, Stephane CHAZELAS <this.addr...@is.invalid>
wrote:[color=blue]
> 2008-04-29, 06:44(-07), Orson:
>[color=green]
> > On Apr 26, 1:34*pm, Stephane CHAZELAS <this.addr...@is.invalid> wrote:[/color]
> [...][color=green][color=darkred]
> >> > 35=D^A58=8f6d5000-8f6d6000:eb840000-f1920000
> >> > 35=E^A58=8f6d5000-8f6d6000:eb840000-f1920000
> >> > 35=8^A58=8f6d5000-8f6d6000:eb840000-f1920000[/color][/color]
> [...][color=green][color=darkred]
> >> grep '\x01' matches "x01".[/color][/color]
>[color=green]
> > Well, grep '\x01' does not return any lines for me given the input
> > above. Does it work for you?[/color]
>
> Well, I can't see any "x01" in that input.
>
> $ echo box01 | grep '\x01'
> box01
>
> works for me.
>
> So does:
>
> $ printf 'xx\1yy\n' | awk '/\1/' | od -c
> 0000000 * x * x 001 * y * y *\n
> 0000006
>
> --
> Stéphane[/color]
A solution for grep is:

grep -P "(35=D\x01)|(35=E)\x01"

The -P will use perl regular expressions and this works. As said in my
original post, the input provided has ^A in place of \x01 (remember
that \x01 is non-printable ascii).

Also, the command:

echo box01 does not produce the SOH char (hex value 0x01). If you pipe
this to file and then open with a hex editor you will see it has the
characters 'b' 'o' 'x' '0' '1'. To produce my test file, I created
java program to output the non-printable ascii character SOH (hex
value 0x01).