grep 'and operator' - SCO

This is a discussion on grep 'and operator' - SCO ; I am stuck. Is there an 'and operator for grep? For example. grep 'first pattern' and 'second pattern' foofile i know you can grep 'first' foofile | grep 'second' but that's a little slow. thanks...

+ Reply to Thread
Results 1 to 10 of 10

Thread: grep 'and operator'

  1. grep 'and operator'

    I am stuck. Is there an 'and operator for grep? For example.


    grep 'first pattern' and 'second pattern' foofile

    i know you can grep 'first' foofile | grep 'second'

    but that's a little slow.


    thanks


  2. Re: grep 'and operator'

    On Mar 2, 9:39 am, "willjay" wrote:
    > I am stuck. Is there an 'and operator for grep? For example.
    >
    > grep 'first pattern' and 'second pattern' foofile
    >
    > i know you can grep 'first' foofile | grep 'second'
    >
    > but that's a little slow.
    >
    > thanks


    I don't know that I buy the premise that the pipeline slows things
    down. Execution speed depends on a lot of things, including file
    sizes, pattern complexity, how many lines are matched by pattern1, the
    OS version, free memory, etc.

    If grep did what you wanted it would have to be able to re-scan a
    line, slowing it down and defeating its use as a filter. For example,
    suppose pattern 1 is "abc" and pattern 2 is "cde". Your piped greps
    have no trouble matching line "abcde." But a single-pass grep after
    matching "abc" would have to back up to the beginning of the line and
    re-scan it for the next pattern.

    If you know that your patterns don't overlap you can avoid a pipe with
    the OR operator:

    ('first pattern'.*'second pattern')|('second pattern'.*'first
    pattern)

    But you may not gain execution speed and you certainly lose clarity.

    --Ray Robert
    Three Star Software


  3. Re: grep 'and operator'

    willjay typed (on Fri, Mar 02, 2007 at 09:39:56AM -0800):
    | I am stuck. Is there an 'and operator for grep? For example.
    |
    |
    | grep 'first pattern' and 'second pattern' foofile
    |
    | i know you can grep 'first' foofile | grep 'second'
    | but that's a little slow.

    grep 'first pattern.*second pattern' foofile

    This will not find lines where the 2nd pattern precedes the 1st one.

    But this will:

    egrep '1st pattern.*2nd pattern|2nd pattern.*1st pattern' foofile

    --
    JP
    ==> http://www.frappr.com/cusm <==

  4. Re: grep 'and operator'

    ThreeStar wrote:
    > On Mar 2, 9:39 am, "willjay" wrote:
    > > I am stuck. Is there an 'and operator for grep? For example.
    > >
    > > grep 'first pattern' and 'second pattern' foofile
    > >
    > > i know you can grep 'first' foofile | grep 'second'
    > >
    > > but that's a little slow.
    > >
    > > thanks

    >
    > I don't know that I buy the premise that the pipeline slows things
    > down. Execution speed depends on a lot of things, including file
    > sizes, pattern complexity, how many lines are matched by pattern1, the
    > OS version, free memory, etc.
    >
    > If grep did what you wanted it would have to be able to re-scan a
    > line, slowing it down and defeating its use as a filter. For example,
    > suppose pattern 1 is "abc" and pattern 2 is "cde". Your piped greps
    > have no trouble matching line "abcde." But a single-pass grep after
    > matching "abc" would have to back up to the beginning of the line and
    > re-scan it for the next pattern.
    >
    > If you know that your patterns don't overlap you can avoid a pipe with
    > the OR operator:
    >
    > ('first pattern'.*'second pattern')|('second pattern'.*'first
    > pattern)
    >
    > But you may not gain execution speed and you certainly lose clarity.
    >
    > --Ray Robert
    > Three Star Software

    I decided to use the pipe. pattern1 occurs in once in very many
    files pattern2 will occur only in one file. I am using grep to find
    the file based upon pattern2 pattern1 returns the lines in the file
    containing pattern2.

    THE line in the file looks like this

    BRP*BLA*BLA*BLA*MOREBLA*YYYYMMDD~

    I am looking for the occurance of BRP\* and \*YYYYMMDD\~


  5. Re: grep 'and operator'

    On Fri, Mar 02, 2007, willjay wrote:
    >ThreeStar wrote:
    >> On Mar 2, 9:39 am, "willjay" wrote:
    >> > I am stuck. Is there an 'and operator for grep? For example.
    >> >
    >> > grep 'first pattern' and 'second pattern' foofile
    >> >
    >> > i know you can grep 'first' foofile | grep 'second'
    >> >
    >> > but that's a little slow.
    >> >
    >> > thanks

    >>

    ....
    >I decided to use the pipe. pattern1 occurs in once in very many
    >files pattern2 will occur only in one file. I am using grep to find
    >the file based upon pattern2 pattern1 returns the lines in the file
    >containing pattern2.
    >
    >THE line in the file looks like this
    >
    >BRP*BLA*BLA*BLA*MOREBLA*YYYYMMDD~
    >
    >I am looking for the occurance of BRP\* and \*YYYYMMDD\~


    This should do the trick if the pattern is on a single line.

    grep -l 'BRP\*.*\*YYYYMMDD"' $files

    If you want to do this with patterns on multiple lines, it's a
    bit trickier. A python program to do this might be:

    #/usr/bin/env python

    import re, sys

    pattern = re.compile(r'BRP\*.*\*YYYYMMDD"', re.DOTALL)

    for file in sys.argv:
    data = open(file).read() # slurp in the file
    if pattern.search(data):
    print file

    sys.exit(0)

    Bill
    --
    INTERNET: bill@Celestial.COM Bill Campbell; Celestial Software LLC
    URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way
    FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676

    ``During times of universal deceit, telling the truth becomes
    a revolutionary act.'' --George Orwell

  6. Re: grep 'and operator'

    Bill Campbell wrote:
    > On Fri, Mar 02, 2007, willjay wrote:
    > >ThreeStar wrote:
    > >> On Mar 2, 9:39 am, "willjay" wrote:
    > >> > I am stuck. Is there an 'and operator for grep? For example.
    > >> >
    > >> > grep 'first pattern' and 'second pattern' foofile
    > >> >
    > >> > i know you can grep 'first' foofile | grep 'second'
    > >> >
    > >> > but that's a little slow.
    > >> >
    > >> > thanks
    > >>

    > ...
    > >I decided to use the pipe. pattern1 occurs in once in very many
    > >files pattern2 will occur only in one file. I am using grep to find
    > >the file based upon pattern2 pattern1 returns the lines in the file
    > >containing pattern2.
    > >
    > >THE line in the file looks like this
    > >
    > >BRP*BLA*BLA*BLA*MOREBLA*YYYYMMDD~
    > >
    > >I am looking for the occurance of BRP\* and \*YYYYMMDD\~

    >
    > This should do the trick if the pattern is on a single line.
    >
    > grep -l 'BRP\*.*\*YYYYMMDD"' $files
    >
    > If you want to do this with patterns on multiple lines, it's a
    > bit trickier. A python program to do this might be:
    >
    > #/usr/bin/env python
    >
    > import re, sys
    >
    > pattern = re.compile(r'BRP\*.*\*YYYYMMDD"', re.DOTALL)
    >
    > for file in sys.argv:
    > data = open(file).read() # slurp in the file
    > if pattern.search(data):
    > print file
    >
    > sys.exit(0)
    >
    > Bill
    > --
    > INTERNET: bill@Celestial.COM Bill Campbell; Celestial Software LLC
    > URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way
    > FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676
    >
    > ``During times of universal deceit, telling the truth becomes
    > a revolutionary act.'' --George Orwell


    what is the -| all about?


  7. Re: grep 'and operator'

    On Mar 2, 6:36 pm, Bill Campbell wrote:
    > On Fri, Mar 02, 2007, willjay wrote:
    > >ThreeStar wrote:
    > >> On Mar 2, 9:39 am, "willjay" wrote:
    > >> > I am stuck. Is there an 'and operator for grep? For example.

    >
    > >> > grep 'first pattern' and 'second pattern' foofile

    >
    > >> > i know you can grep 'first' foofile | grep 'second'

    >
    > >> > but that's a little slow.

    >
    > >> > thanks

    >
    > ...
    > >I decided to use the pipe. pattern1 occurs in once in very many
    > >files pattern2 will occur only in one file. I am using grep to find
    > >the file based upon pattern2 pattern1 returns the lines in the file
    > >containing pattern2.

    >
    > >THE line in the file looks like this

    >
    > >BRP*BLA*BLA*BLA*MOREBLA*YYYYMMDD~

    >
    > >I am looking for the occurance of BRP\* and \*YYYYMMDD\~

    >
    > This should do the trick if the pattern is on a single line.
    >
    > grep -l 'BRP\*.*\*YYYYMMDD"' $files
    >
    > If you want to do this with patterns on multiple lines, it's a
    > bit trickier. A python program to do this might be:
    >
    > #/usr/bin/env python
    >
    > import re, sys
    >
    > pattern = re.compile(r'BRP\*.*\*YYYYMMDD"', re.DOTALL)
    >
    > for file in sys.argv:
    > data = open(file).read() # slurp in the file
    > if pattern.search(data):
    > print file
    >
    > sys.exit(0)
    >
    > Bill
    > --
    > INTERNET: b...@Celestial.COM Bill Campbell; Celestial Software LLC
    > URL:http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way
    > FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676
    >
    > ``During times of universal deceit, telling the truth becomes
    > a revolutionary act.'' --George Orwell- Hide quoted text -
    >
    > - Show quoted text -


    sorry i was reading on my Blackberry and the -l looke like -(pipe),
    understood.


  8. Re: grep 'and operator'


    ----- Original Message -----
    From: "willjay"
    Newsgroups: comp.unix.sco.misc
    To:
    Sent: Friday, March 02, 2007 5:45 PM
    Subject: Re: grep 'and operator'


    > ThreeStar wrote:
    >> On Mar 2, 9:39 am, "willjay" wrote:
    >> > I am stuck. Is there an 'and operator for grep? For example.
    >> >
    >> > grep 'first pattern' and 'second pattern' foofile
    >> >
    >> > i know you can grep 'first' foofile | grep 'second'
    >> >
    >> > but that's a little slow.
    >> >
    >> > thanks

    >>
    >> I don't know that I buy the premise that the pipeline slows things
    >> down. Execution speed depends on a lot of things, including file
    >> sizes, pattern complexity, how many lines are matched by pattern1, the
    >> OS version, free memory, etc.
    >>
    >> If grep did what you wanted it would have to be able to re-scan a
    >> line, slowing it down and defeating its use as a filter. For example,
    >> suppose pattern 1 is "abc" and pattern 2 is "cde". Your piped greps
    >> have no trouble matching line "abcde." But a single-pass grep after
    >> matching "abc" would have to back up to the beginning of the line and
    >> re-scan it for the next pattern.
    >>
    >> If you know that your patterns don't overlap you can avoid a pipe with
    >> the OR operator:
    >>
    >> ('first pattern'.*'second pattern')|('second pattern'.*'first
    >> pattern)
    >>
    >> But you may not gain execution speed and you certainly lose clarity.
    >>
    >> --Ray Robert
    >> Three Star Software

    > I decided to use the pipe. pattern1 occurs in once in very many
    > files pattern2 will occur only in one file. I am using grep to find
    > the file based upon pattern2 pattern1 returns the lines in the file
    > containing pattern2.
    >
    > THE line in the file looks like this
    >
    > BRP*BLA*BLA*BLA*MOREBLA*YYYYMMDD~
    >
    > I am looking for the occurance of BRP\* and \*YYYYMMDD\~


    If pattern1 and pattern 2 are always on the same line, and pattern2 always
    comes after pattern1, then really this is just one pattern and one grep
    command.

    grep "BRP\*.*\*YYYMMDD\~"

    If BRP always appears at the beginning of the line, or rather, the BRP you
    are interested in, similarly if the date always appears at the end of the
    line, then you may want to include beginning/ending of line anchors in the
    pattern so that if BRP* just happens to appear in the data somewhere it
    won't get picked up as a match that you don't want.

    grep '^BRP\*.*\*YYYMMDD\~$'

    If YYMMDD is really variable data you can get away with double quotes in
    this case to allow variable expansion
    D=`date +%Y%m%d`
    grep "^BRP\*.*\*${D}\~$"

    or

    grep "^BRP\*.*\*`date +%Y%m%d`\~$"

    Your pipeline would find lines where YYYMMDD appeared before BRP if any
    existed.
    But since this looks like an EDI transmission you know the format of the
    lines doesn't change except according to specifically defined rules.

    And, also if this is an EDI transmission, there is usually no reason to
    expect line-feeds to appear at the ends of records. The ~ might be the
    record delimiter and so you may want to switch to awk where you can specify
    field an record delimiters at will and thus avoid having grep choke trying
    to read an entire large file as one line, and/or, uselessly returning the
    whole file as a match.

    This sets filed delimiter to * and record delimiter to ~ and matches lines
    where field 1 is BRP and field 7 is YYYMMDD
    awk -F '*' 'BEGIN{RS="~"}($1=="BRP")&&($7=="YYYMMDD"){print $0}'

    If YYYMMDD really needs to be some variable data:
    awk -F '*' 'BEGIN{RS="~"}($1=="BRP")&&($7==$D){print $0}' D=`date +%Y%m%d`

    If the tilde is not the record delimiter after all and linefeed is:
    awk -F '*' '($1=="BRP")&&($7==$D){print $0}' D=`date +%Y%m%d`

    If what you're really interested in is the contents of some other field(s)
    in the matching record
    This will print just field 3, 4, & 5, without the *'s
    awk -F '*' '($1=="BRP")&&($7==$D){print $3,$4,$5}' D=`date +%Y%m%d`

    Or if you just wanted to know the name of the file that has a matching line,
    and there are linefeeds between records, and the patterns will always appear
    in the same order on the line, then the grep is the fastest.

    I could guess your real intentions if this, if that... all day.
    As you can see there are many ways to get exactly what you might be looking
    for, so if you can describe what you _really_ need to do instead of trying
    to generalize too much and describe what you think you need, you might get
    an answer that will be more robust & reliable (less chance of false
    positives or negatives). And possibly easier to modify further on your own
    after you get the first working example.

    The awk commands above, although they may not look it at first glance, are
    simpler to grasp than a regular expression would be (a la grep) to do the
    same thing, and are more exact/specific to boot. They are actually reading
    the record much the way the record was written by some database process, and
    the way some other database process is going to read them. By setting the
    record and field delimiters to what they really are, and then working with
    fields instead of trying to work with the whole line as one chunk of data
    the way grep has to.
    Also, if it turns out later that you need to modify the matching rule
    further, like say, "match as above, except not if field 3 is void", you
    could add yet another grep to the pipe, or in the awk case, even if you
    couldn't have written the awk commands above, I bet you have a fair stab at
    modifying them successfully.

    original:
    awk -F '*' '($1=="BRP")&&($7==$D){print $3,$4,$5}' D=`date +%Y%m%d`

    new:
    awk -F '*' '($1=="BRP")&&($3!="void")&&($7==$D){print $3,$4,$5}' D=`date
    +%Y%m%d`


    Brian K. White -- brian@aljex.com -- http://www.aljex.com/bkw/
    +++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
    filePro BBx Linux SCO FreeBSD #callahans Satriani Filk!



  9. Re: grep 'and operator'

    On Mar 2, 9:39 am, "willjay" wrote:
    >> I am stuck. Is there an 'and operator for grep? For example.
    >>
    >> grep 'first pattern' and 'second pattern' foofile


    agrep had an 'and' operator: agrep 'first pattern;second pattern' foofile

    An agrep built for OSR5 is available at: ftp://ftp.armory.com/pub/scobins/agrep
    Man page: http://www.armory.com:457/cgi-bin/man/man?agrep

    John
    --
    John DuBois spcecdt@armory.com KC6QKZ/AE http://www.armory.com/~spcecdt/

  10. Re: grep 'and operator' - with xargs

    I am not too sure how you guys do ANDing in grep (exp1 and exp2 and...) on SCO UNIX.

    But for GNU/Linux (eg RHEL), this is a possible robust & efficient solution:

    find /path/to/files/ -print0 |xargs -0 grep -l 'exp 1' |xargs -i{} grep -l 'exp 2' "{}"|xargs -i{} grep -l 'exp 3' "{}"

    - 1 liner solution.
    - Supports file names with spaces
    - Supports exp with spaces.
    - Supports ANDing multiple expressions.
    - Should be efficient processing because xargs is used.

+ Reply to Thread