[flex+bison] Multiple parsers+lexers / one application - Linux

This is a discussion on [flex+bison] Multiple parsers+lexers / one application - Linux ; Hi all, I would like to code an application that can parse several input formats (which are semantically equivalent but with different lexemes and grammar). Imagine that I can detect in the main() function the kind of language to expect ...

+ Reply to Thread
Results 1 to 5 of 5

Thread: [flex+bison] Multiple parsers+lexers / one application

  1. [flex+bison] Multiple parsers+lexers / one application

    Hi all,

    I would like to code an application that can parse several input formats
    (which are semantically equivalent but with different lexemes and grammar).

    Imagine that I can detect in the main() function the kind of language to
    expect (e.g. by looking at the file extension). How should I manage with
    flex and bison to be able to load several parser/lexer depending on the
    input format I have detected ?

    For now, I have vaguely heard about two options:

    - Prefix yyparse() and yylex() functions by the type of the language.
    But I don't really know how to do this...

    - Dynamically load a library that handle the lexing/parsing through an API.

    Any information, comment, tutorial, or piece of code helping to
    implement such a scheme is welcome.

    Regards
    --
    Emmanuel Fleury | Office: 261
    Associate Professor, | Phone: +33 (0)5 40 00 69 34
    LaBRI, Domaine Universitaire | Fax: +33 (0)5 40 00 66 69
    351, Cours de la Libération | email: emmanuel.fleury@labri.fr
    33405 Talence Cedex, France | URL: http://www.labri.fr/~fleury

  2. Re: [flex+bison] Multiple parsers+lexers / one application

    Emmanuel Fleury wrote:
    >
    > I would like to code an application that can parse several input formats
    > (which are semantically equivalent but with different lexemes and grammar).
    >
    > Imagine that I can detect in the main() function the kind of language to
    > expect (e.g. by looking at the file extension). How should I manage with
    > flex and bison to be able to load several parser/lexer depending on the
    > input format I have detected ?
    >
    > For now, I have vaguely heard about two options:
    >
    > - Prefix yyparse() and yylex() functions by the type of the language.
    > But I don't really know how to do this...
    >
    > - Dynamically load a library that handle the lexing/parsing through an API.
    >
    > Any information, comment, tutorial, or piece of code helping to
    > implement such a scheme is welcome.


    Doh, I got it:

    Multiple parsers and lexers

    It used to be that using multiple parsers within a single program
    required a fair amount of careful tuning of the files generated by lex
    and yacc. Conveniently, this has been corrected: modern versions of both
    lex and yacc allow you to specify a prefix to use on the names of
    symbols in generated code. Just do that -- it's easier. In flex, the
    option is -Pprefix; in modern yacc, it's most often -p prefix. You can
    almost always arrange to use flex or bison, if the default lex and yacc
    available to you don't support this.

    Seen on: http://www-128.ibm.com/developerwork...l-lexyac2.html

    I guess this is the way to go.

    Regards
    --
    Emmanuel Fleury | Office: 261
    Associate Professor, | Phone: +33 (0)5 40 00 69 34
    LaBRI, Domaine Universitaire | Fax: +33 (0)5 40 00 66 69
    351, Cours de la Libération | email: emmanuel.fleury@labri.fr
    33405 Talence Cedex, France | URL: http://www.labri.fr/~fleury

  3. Re: [flex+bison] Multiple parsers+lexers / one application

    Emmanuel Fleury wrote:
    > It used to be that using multiple parsers within a single program
    > required a fair amount of careful tuning of the files generated by lex
    > and yacc. Conveniently, this has been corrected: modern versions of both
    > lex and yacc allow you to specify a prefix to use on the names of
    > symbols in generated code. Just do that -- it's easier. In flex, the
    > option is -Pprefix; in modern yacc, it's most often -p prefix. You can
    > almost always arrange to use flex or bison, if the default lex and yacc
    > available to you don't support this.


    > Seen on: http://www-128.ibm.com/developerwork...l-lexyac2.html


    > I guess this is the way to go.


    I have been doing it like this since several years now (even token-
    izing/parsing parts of a single file files with different flex/bison
    generated parsers) and didn't have any problems with it. The only
    thing to take care of is that the -P option of flex does not allow
    spaces between the '-P' and the name for the replacement for 'yy'.

    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

  4. Re: [flex+bison] Multiple parsers+lexers / one application

    Emmanuel Fleury wrote:
    > option is -Pprefix; in modern yacc, it's most often -p prefix. You can
    > almost always arrange to use flex or bison, if the default lex and yacc
    > available to you don't support this.


    Berkeley yacc is portable, and supports this.

    http://invisible-island.net/byacc/

    --
    Thomas E. Dickey
    http://invisible-island.net
    ftp://invisible-island.net

  5. Re: [flex+bison] Multiple parsers+lexers / one application

    Emmanuel Fleury wrote:
    > Hi all,


    > I would like to code an application that can parse several input formats
    > (which are semantically equivalent but with different lexemes and grammar).


    > Imagine that I can detect in the main() function the kind of language to
    > expect (e.g. by looking at the file extension). How should I manage with
    > flex and bison to be able to load several parser/lexer depending on the
    > input format I have detected ?


    > For now, I have vaguely heard about two options:


    > - Prefix yyparse() and yylex() functions by the type of the language.
    > But I don't really know how to do this...


    > - Dynamically load a library that handle the lexing/parsing through an API.


    vile does this (configure --builtin-filters vs --loadable-filters).
    It uses both approaches to compile (mostly) lex/flex-based parsers
    which can be separately compiled, linked-in, or loaded dynamically.

    See
    http://invisible-island.net/vile/

    --
    Thomas E. Dickey
    http://invisible-island.net
    ftp://invisible-island.net

+ Reply to Thread