flex+bison problem: syntax error that shouldn't be - Unix

This is a discussion on flex+bison problem: syntax error that shouldn't be - Unix ; I've just started looking into flex+bison and unfortunately I've stumbled on a problem that I'm not able to figure out. I have a grammar definition which I believe is sane and a lexer which works fine but when I try ...

+ Reply to Thread
Results 1 to 6 of 6

Thread: flex+bison problem: syntax error that shouldn't be

  1. flex+bison problem: syntax error that shouldn't be

    I've just started looking into flex+bison and unfortunately I've stumbled
    on a problem that I'm not able to figure out. I have a grammar definition
    which I believe is sane and a lexer which works fine but when I try to
    feed the parser a valid entry, the parser complains about a syntax error.

    Could anyone please point out to me what is wrong with this code? Thanks
    in advance.

    The code is as follows:

    the flex source code:

    %{
    #include
    #include

    #include "definition.h"
    extern char* yylval;
    %}
    char [A-Za-z]
    num [0-9]
    eq [=]
    name {char}+
    age {num}+
    %%
    {name} { yylval = strdup(yytext); return NAME; }
    {eq} { return EQ; }
    {age} { yylval = strdup(yytext); return AGE; }
    [ \t] /* ignore */
    .. {yyerror("invalid character\n");}
    %%

    int yywrap()
    {
    return 1;
    }


    The bison source code:
    %{
    #include

    typedef char* string;
    #define YYSTYPE string
    %}

    %token NAME EQ AGE

    %start file

    %%
    file: record file
    | record
    ;
    record: '\n'
    | NAME EQ AGE { printf("%s is %s years old!!!\n", $1, $3); }
    ;
    %%

    int main(void)
    {
    yyparse();
    return 0;
    }

    int yyerror(char *msg)
    {
    yydebug=1;
    printf("Error encountered: %s \n", msg);
    }

  2. Re: flex+bison problem: syntax error that shouldn't be

    On Aug 10, 1:15 pm, Rui Maciel wrote:
    > I've just started looking into flex+bison and unfortunately I've stumbled
    > on a problem that I'm not able to figure out. I have a grammar definition
    > which I believe is sane and a lexer which works fine but when I try to
    > feed the parser a valid entry, the parser complains about a syntax error.
    >
    > Could anyone please point out to me what is wrong with this code? Thanks
    > in advance.
    >
    > The code is as follows:
    >
    > the flex source code:
    >
    > %{
    > #include
    > #include
    >
    > #include "definition.h"

    [...]

    You didn't include definition.h, so I can't compile this to try it.

    Also, please include an example of some input that demonstrates the
    problem, and say what error you get.

  3. Re: flex+bison problem: syntax error that shouldn't be

    On Sun, 10 Aug 2008 23:53:38 -0700, fjblurt wrote:

    >> #include "definition.h"

    > [...]
    >
    > You didn't include definition.h, so I can't compile this to try it.


    You are right. Here it is:

    #ifndef DEFINITION
    #define DEFINITION

    #define NAME 257
    #define EQ 258
    #define AGE 259

    #endif


    > Also, please include an example of some input that demonstrates the
    > problem, and say what error you get.


    With the following input it appears that the parser simply chokes at the
    first token (NAME):

    joe=10

    On the other hand, it appears that the parser doesn't complain, at least
    initially, with input such as:

    =10

    which, to my newbie brain, doesn't make sense.


    So, can anyone help?

    Rui Maciel

  4. Re: flex+bison problem: syntax error that shouldn't be

    Rui Maciel wrote:
    > I've just started looking into flex+bison and unfortunately I've stumbled
    > on a problem that I'm not able to figure out. I have a grammar definition
    > which I believe is sane and a lexer which works fine but when I try to
    > feed the parser a valid entry, the parser complains about a syntax error.


    > Could anyone please point out to me what is wrong with this code? Thanks
    > in advance.


    > The code is as follows:


    > the flex source code:


    > %{
    > #include
    > #include


    > #include "definition.h"
    > extern char* yylval;
    > %}
    > char [A-Za-z]
    > num [0-9]
    > eq [=]
    > name {char}+
    > age {num}+
    > %%
    > {name} { yylval = strdup(yytext); return NAME; }
    > {eq} { return EQ; }
    > {age} { yylval = strdup(yytext); return AGE; }
    > [ \t] /* ignore */
    > . {yyerror("invalid character\n");}
    > %%


    > int yywrap()
    > {
    > return 1;
    > }



    > The bison source code:
    > %{
    > #include


    > typedef char* string;
    > #define YYSTYPE string
    > %}


    > %token NAME EQ AGE


    > %start file


    > %%
    > file: record file
    > | record
    > ;
    > record: '\n'
    > | NAME EQ AGE { printf("%s is %s years old!!!\n", $1, $3); }
    > ;
    > %%


    > int main(void)
    > {
    > yyparse();
    > return 0;
    > }


    > int yyerror(char *msg)
    > {
    > yydebug=1;
    > printf("Error encountered: %s \n", msg);
    > }


    It's rather unclear what you consider to be not working.
    The "grammar" you have here is rather simple, all it
    accepts as correct syntax is something like

    [ \t]?[A-Za-z]+[ \t]?=[ \t]?[0-9]+

    and repetititons of this, e.g. "xyz = 23 abc = 123". Everything
    else is a syntax error - that's how the grammar is specified.

    I also would recomment to change the whole thing a bit.
    NAME looks like it is supposed to be string, but AGE
    rather likely should be an (unsigned) integer value (and
    you're probably going to use that value as in integer
    sometime later). So I would recommend to change the
    files a bit:

    scanner.l
    ----------8<--------------------------
    %option noyywrap nounput

    %{
    #include
    #include
    #include "definition.h"
    extern void yyerror( const char *msg );
    %}

    char [A-Za-z]
    num [0-9]
    eq [=]
    name {char}+
    age {num}+

    %%
    {name} { yylval.sptr = strdup( yytext );
    return NAME; }
    {eq} { return EQ; }
    {age} { yylval.ulval = strtoul( yytext, NULL, 10 );
    return AGE; }
    [ \t] /* ignore */
    .. { yyerror( "invalid character\n" ); }
    %%
    ----------8<--------------------------

    definition.y:
    ----------8<--------------------------
    %{
    #include

    void yyerror( const char *msg );
    extern int yylex( void );
    %}

    %union {
    char * sptr;
    unsigned long int ulval;
    }

    %token NAME
    %token AGE
    %token EQ

    %start file

    %%
    file: record file
    | record
    ;
    record: '\n'
    | NAME EQ AGE { printf( "%s is %lu years old!!!\n", $1, $3 );
    free( $1 ); }
    ;
    %%

    int
    main( void )
    {
    yyparse( );
    return 0;
    }

    void
    yyerror( const char *msg )
    {
    yydebug = 1;
    printf( "Error encountered: %s \n", msg );
    }
    ----------8<--------------------------

    Create the program with

    > bison -d -v -t -o definition.c definition.y
    > flex -B -o scanner.c scanner.l
    > gcc -W -Wall -o my_grammar scanner.c definition.c


    This should compiler cleanly and create a file 'definition.h'
    to be included in scanner.l.
    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

  5. Re: flex+bison problem: syntax error that shouldn't be

    On Mon, 11 Aug 2008 10:37:00 +0000, Jens Thoms Toerring wrote:

    > It's rather unclear what you consider to be not working. The "grammar"
    > you have here is rather simple, all it accepts as correct syntax is
    > something like
    >
    > [ \t]?[A-Za-z]+[ \t]?=[ \t]?[0-9]+
    >
    > and repetititons of this, e.g. "xyz = 23 abc = 123". Everything else is
    > a syntax error - that's how the grammar is specified.


    Indeed. I was expecting to get the behavior that you described but the
    parser simply wasn't cooperating. The code that I presented, when
    compiled without specifying any flags (flex lex.l, bison -t grammar.y),
    produced this result:

    rui@kubuntu:flex$ ./main
    joe=10
    Error encountered: syntax error
    Cleanup: discarding lookahead token $undefined ()
    Stack now 0

    On the other hand, your code appears to work exactly as expected. Do you
    happen to know what went wrong with my code so that it could compile
    flawlessly but nonetheless produced broken code?


    Thanks for the help,
    Rui Maciel

  6. Re: flex+bison problem: syntax error that shouldn't be

    Rui Maciel wrote:
    > On Mon, 11 Aug 2008 10:37:00 +0000, Jens Thoms Toerring wrote:


    > > It's rather unclear what you consider to be not working. The "grammar"
    > > you have here is rather simple, all it accepts as correct syntax is
    > > something like
    > >
    > > [ \t]?[A-Za-z]+[ \t]?=[ \t]?[0-9]+
    > >
    > > and repetititons of this, e.g. "xyz = 23 abc = 123". Everything else is
    > > a syntax error - that's how the grammar is specified.


    > Indeed. I was expecting to get the behavior that you described but the
    > parser simply wasn't cooperating. The code that I presented, when
    > compiled without specifying any flags (flex lex.l, bison -t grammar.y),
    > produced this result:


    > rui@kubuntu:flex$ ./main
    > joe=10
    > Error encountered: syntax error
    > Cleanup: discarding lookahead token $undefined ()
    > Stack now 0


    > On the other hand, your code appears to work exactly as expected. Do you
    > happen to know what went wrong with my code so that it could compile
    > flawlessly


    Flawlessly is a bit of a msinomer, the compiler complained quite
    a bit about several things;-)

    >but nonetheless produced broken code?


    Well, the content of the header file you posted in your reply
    to "fjblurt"

    #ifndef DEFINITION
    #define DEFINITION

    #define NAME 257
    #define EQ 258
    #define AGE 259

    #endif

    doesn't look correct when compared to what I get when I run bison
    on your .y file. In the resulting .tab.c file the definintions
    look a bit different

    #define NAME 258
    #define EQ 259
    #define AGE 260

    And that could explain the problem you're getting since that makes
    the tokenizer and lexer misunderstanding each other. If I use the
    definitions from the .tab.c file everything starts to work again.
    It tends to be a good idea to use the header file created by bison
    when the '-d' option is specified because that will help to avoid
    such problems.
    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

+ Reply to Thread