flex, bison: compilation warning - Unix

This is a discussion on flex, bison: compilation warning - Unix ; I've just started to learn how to use flex and bison and on my first working code, the compilation threw the following warning: rui@kubuntu:flex$ make flex -B -o scanner.c scanner.l bison -d -v -t -o definition.c definition.y gcc -W -Wall ...

+ Reply to Thread
Results 1 to 10 of 10

Thread: flex, bison: compilation warning

  1. flex, bison: compilation warning

    I've just started to learn how to use flex and bison and on my first
    working code, the compilation threw the following warning:

    rui@kubuntu:flex$ make
    flex -B -o scanner.c scanner.l
    bison -d -v -t -o definition.c definition.y
    gcc -W -Wall -o my_grammar scanner.c definition.c
    scanner.c: In function ‘yy_get_next_buffer’:
    scanner.c:984: warning: comparison between signed and unsigned


    As I haven't specified any unsigned data type in my code and I haven't
    found anything related to this problem, I don't have a clue regarding the
    cause of this problem and how to fix it.

    So, can anyone help?


    Best regards,
    Rui Maciel

  2. Re: flex, bison: compilation warning

    On Aug 23, 7:00 am, Rui Maciel wrote:
    > I've just started to learn how to use flex and bison and on my first
    > working code, the compilation threw the following warning:
    >
    > rui@kubuntu:flex$ make
    > flex -B -o scanner.c scanner.l
    > bison -d -v -t -o definition.c definition.y
    > gcc -W -Wall -o my_grammar scanner.c definition.c
    > scanner.c: In function yy_get_next_buffer:
    > scanner.c:984: warning: comparison between signed and unsigned
    >
    > As I haven't specified any unsigned data type in my code and I haven't
    > found anything related to this problem, I don't have a clue regarding the
    > cause of this problem and how to fix it.


    It's quite possible it is not really a problem. The compiler warns
    about this because mixing signed and unsigned types *can* be a bug,
    but it can also be harmless.

    For example, if I wrote the following version of memset():

    void my_memset(char *p, int v, unsigned len) {
    int i;
    for (i = 0; i < len; i++)
    p[i] = v;
    }

    the compiler will issue such a warning. Suppose I should happen to
    call my_memset with len == 3000000000 (assuming int and unsigned are
    32 bits). `i' is an int and can only take values between -2147483648
    and 2147483647, so the test `i < len' will always be true. `i' will
    be incremented up to 2147483647, then wrap around to negative values
    and I'll start setting memory that I didn't intend to. Moreover, the
    loop will never terminate.

    However, if I never call my_memset with such a large value for len, or
    somewhere else I have a test for such cases, then everything will be
    fine.

    So there is probably some code like that in the generated scanner.c.
    It could be because of something you wrote in scanner.l, or it could
    be a bug that flex generated such code, or it could be that it is
    actually harmless. To be sure, you'd have to look at the code in
    scanner.c (warning, it may not be easy to read) and see what is going
    on. You could also post your scanner.l and tell us what version of
    flex you are using, and maybe someone else can see what's going on.

    If this were a "real" program, it would be important to figure out the
    cause of this warning. However, for a simple program you're writing
    as an exercise, I will bet that whether or not it's really a bug, it
    won't have any effect on the things you try. If it were me, I would
    probably ignore the warning and just concentrate on learning about how
    the program works.

  3. Re: flex, bison: compilation warning

    Rui Maciel wrote:
    > I've just started to learn how to use flex and bison and on my first
    > working code, the compilation threw the following warning:


    > rui@kubuntu:flex$ make
    > flex -B -o scanner.c scanner.l
    > bison -d -v -t -o definition.c definition.y
    > gcc -W -Wall -o my_grammar scanner.c definition.c
    > scanner.c: In function ‘yy_get_next_buffer’:
    > scanner.c:984: warning: comparison between signed and unsigned


    > As I haven't specified any unsigned data type in my code and I haven't
    > found anything related to this problem, I don't have a clue regarding the
    > cause of this problem and how to fix it.


    bison generates a lot of code for you and the version of bison
    you're using obviously creates code that contains a comparison
    between a signed and an unsigned integer. Just look at the code
    at line 984 in the generated file scanner.c to see what exactly
    is happening there. It's rather likely not a serious problem. If
    the warning bothers you try to compile with '-Wno-sign-compare'
    or check if there's a newer version of bison that may not have
    this issue.
    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

  4. Re: flex, bison: compilation warning

    Jens Thoms Toerring wrote:
    > Rui Maciel wrote:
    > > I've just started to learn how to use flex and bison and on my first
    > > working code, the compilation threw the following warning:


    > > rui@kubuntu:flex$ make
    > > flex -B -o scanner.c scanner.l
    > > bison -d -v -t -o definition.c definition.y
    > > gcc -W -Wall -o my_grammar scanner.c definition.c
    > > scanner.c: In function ‘yy_get_next_buffer’:
    > > scanner.c:984: warning: comparison between signed and unsigned


    > > As I haven't specified any unsigned data type in my code and I haven't
    > > found anything related to this problem, I don't have a clue regarding the
    > > cause of this problem and how to fix it.


    > bison generates a lot of code for you and the version of bison


    That was meast to be flex, not bison!

    > you're using obviously creates code that contains a comparison
    > between a signed and an unsigned integer. Just look at the code
    > at line 984 in the generated file scanner.c to see what exactly
    > is happening there. It's rather likely not a serious problem. If
    > the warning bothers you try to compile with '-Wno-sign-compare'
    > or check if there's a newer version of bison that may not have
    > this issue.


    And here again...

    BTW, is it coming from at a line where YY_INPUT is called? YY_INPUT
    is a macro where you find

    size_t n;
    for ( n = 0; n < max_size &&...

    Check what is passed to the macro as 'max_size', it sometimes is
    an int instead of an unsigned int and thus the warning (newer
    versions of flex don't seem to have that problem anymore).

    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

  5. Re: flex, bison: compilation warning

    Rui Maciel writes:
    > I've just started to learn how to use flex and bison and on my first
    > working code, the compilation threw the following warning:
    >
    > rui@kubuntu:flex$ make
    > flex -B -o scanner.c scanner.l
    > bison -d -v -t -o definition.c definition.y
    > gcc -W -Wall -o my_grammar scanner.c definition.c
    > scanner.c: In function yy_get_next_buffer:
    > scanner.c:984: warning: comparison between signed and unsigned
    >
    >
    > As I haven't specified any unsigned data type in my code and I haven't
    > found anything related to this problem, I don't have a clue regarding the
    > cause of this problem and how to fix it.


    This isn't a problem and the solution is to ignore the warning (there
    is little point in messing around with generated code because it
    causes spurious compiler warnings). Comparing values stored in objects
    declared to have signed and unsigned type is a completely ordinary
    C-operation and its semantics (in this respect) are defined in
    6.3.1.8.1|, 'usual arithmetic conversions'.

  6. Re: flex, bison: compilation warning

    On Sat, 23 Aug 2008 16:30:57 +0000, Jens Thoms Toerring wrote:

    > bison generates a lot of code for you and the version of bison you're
    > using obviously creates code that contains a comparison between a signed
    > and an unsigned integer. Just look at the code at line 984 in the
    > generated file scanner.c to see what exactly is happening there.


    I've looked into scanner.c and that particular line consists of:


    /* Read in more data. */
    --> YY_INPUT( (&YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[number_to_move]),
    (yy_n_chars), (size_t) num_to_read );



    That doesn't say much but YY_INPUT is a macro that is replaced by some
    code that although it has a few comparisons they all involve, as far as I
    can tell, signed types.

    The macro code is as follows:


    /* Gets input and stuffs it into "buf". number of characters read, or
    YY_NULL,
    * is returned in "result".
    */
    #ifndef YY_INPUT
    #define YY_INPUT(buf,result,max_size) \
    if ( YY_CURRENT_BUFFER_LVALUE->yy_is_interactive ) \
    { \
    int c = '*'; \
    int n; \
    for ( n = 0; n < max_size && \
    (c = getc( yyin )) != EOF && c != '\n'; +
    +n ) \
    buf[n] = (char) c; \
    if ( c == '\n' ) \
    buf[n++] = (char) c; \
    if ( c == EOF && ferror( yyin ) ) \
    YY_FATAL_ERROR( "input in flex scanner failed" );
    \
    result = n; \
    } \
    else \
    { \
    errno=0; \
    while ( (result = fread(buf, 1, max_size, yyin))==0 &&
    ferror(yyin)) \
    { \
    if( errno != EINTR) \
    { \
    YY_FATAL_ERROR( "input in flex scanner
    failed" ); \
    break; \
    } \
    errno=0; \
    clearerr(yyin); \
    } \
    }\
    \

    #endif


    So can anyone


    > It's
    > rather likely not a serious problem. If the warning bothers you try to
    > compile with '-Wno-sign-compare' or check if there's a newer version of
    > bison that may not have this issue.


    I see what you mean. Nonetheless, I don't believe that an automated code
    generator, specially a fundamental tool like GNU flex, inadvertently
    produces code which is less than good.


    Rui Maciel

  7. Re: flex, bison: compilation warning

    Rui Maciel writes:
    > On Sat, 23 Aug 2008 16:30:57 +0000, Jens Thoms Toerring wrote:


    [...]

    >> It's
    >> rather likely not a serious problem. If the warning bothers you try to
    >> compile with '-Wno-sign-compare' or check if there's a newer version of
    >> bison that may not have this issue.

    >
    > I see what you mean. Nonetheless, I don't believe that an automated code
    > generator, specially a fundamental tool like GNU flex, inadvertently
    > produces code which is less than good.


    And that is wrong. gcc prints a lot of warnings whose quality
    falls between 'less than good' and 'completely useless'.

  8. Re: flex, bison: compilation warning

    On Aug 24, 9:17 am, Rui Maciel wrote:
    > On Sat, 23 Aug 2008 16:30:57 +0000, Jens Thoms Toerring wrote:
    > > bison generates a lot of code for you and the version of bison you're
    > > using obviously creates code that contains a comparison between a signed
    > > and an unsigned integer. Just look at the code at line 984 in the
    > > generated file scanner.c to see what exactly is happening there.

    >
    > I've looked into scanner.c and that particular line consists of:
    >
    >
    > /* Read in more data. */
    > --> YY_INPUT( (&YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[number_to_move]),
    > (yy_n_chars), (size_t) num_to_read );
    >

    >
    > That doesn't say much but YY_INPUT is a macro that is replaced by some
    > code that although it has a few comparisons they all involve, as far as I
    > can tell, signed types.
    >
    > The macro code is as follows:
    >
    >
    > /* Gets input and stuffs it into "buf". number of characters read, or
    > YY_NULL,
    > * is returned in "result".
    > */
    > #ifndef YY_INPUT
    > #define YY_INPUT(buf,result,max_size) \
    > if ( YY_CURRENT_BUFFER_LVALUE->yy_is_interactive ) \
    > { \
    > int c = '*'; \
    > int n; \
    > for ( n = 0; n < max_size && \

    ^^^^^^^^^^^^
    That's it. YY_INPUT is called with max_size = (size_t)num_to_read.
    size_t is an unsigned type.

    So if it is possible that num_to_read (after the cast) were larger
    than the largest int, then this is a bug. However, it might be that
    that can only happen if you have a token which is 2^31 characters long
    or something, which would probably fail for other reasons.



  9. Re: flex, bison: compilation warning

    fjblurt@yahoo.com writes:
    > On Aug 24, 9:17 am, Rui Maciel wrote:
    >> On Sat, 23 Aug 2008 16:30:57 +0000, Jens Thoms Toerring wrote:
    >> > bison generates a lot of code for you and the version of bison you're
    >> > using obviously creates code that contains a comparison between a signed
    >> > and an unsigned integer. Just look at the code at line 984 in the
    >> > generated file scanner.c to see what exactly is happening there.

    >>
    >> I've looked into scanner.c and that particular line consists of:
    >>
    >>
    >> /* Read in more data. */
    >> --> YY_INPUT( (&YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[number_to_move]),
    >> (yy_n_chars), (size_t) num_to_read );
    >>

    >>
    >> That doesn't say much but YY_INPUT is a macro that is replaced by some
    >> code that although it has a few comparisons they all involve, as far as I
    >> can tell, signed types.
    >>
    >> The macro code is as follows:
    >>
    >>
    >> /* Gets input and stuffs it into "buf". number of characters read, or
    >> YY_NULL,
    >> * is returned in "result".
    >> */
    >> #ifndef YY_INPUT
    >> #define YY_INPUT(buf,result,max_size) \
    >> if ( YY_CURRENT_BUFFER_LVALUE->yy_is_interactive ) \
    >> { \
    >> int c = '*'; \
    >> int n; \
    >> for ( n = 0; n < max_size && \

    > ^^^^^^^^^^^^
    > That's it. YY_INPUT is called with max_size = (size_t)num_to_read.
    > size_t is an unsigned type.
    >
    > So if it is possible that num_to_read (after the cast) were larger
    > than the largest int, then this is a bug.


    Nope. The relevant part of the C-standard states that

    Otherwise, if the type of the operand with signed integer type
    can represent all of the values of the type of the operand
    with unsigned integer type, then the operand with unsigned
    integer type is converted to the type of the operand with
    signed integer type.

    Otherwise, both operands are converted to the unsigned integer
    type corresponding to the type of the operand with signed
    integer type.

    Because int cannot (usually) represent all values of the type hiding
    behind size_t, both operands will be converted to size_t before doing
    the comparison. The 'theoretical' problem situation is here that, if
    the value of n could be negative, this conversion would yield a large,
    unsigned value and the <-test could then be false despite (naive)
    people might assume that a negative number should be smaller than any
    possible unsigned value. But taking the way unsigned types in C are
    defined to 'work' into account, this assumption is simply wrong and
    people who cannot get their heads around that have basic problems
    understanding the difference between 'integer arithmetic defined on
    mod n' and 'integer arithmetic defined on the set of integral
    numbers'. The way to 'fix' this would be an introductory course on
    linear algebra, not a compiler warning.

  10. Re: flex, bison: compilation warning

    Rainer Weikusat writes:

    [...]

    > The 'theoretical' problem situation is here that, if
    > the value of n could be negative, this conversion would yield a large,
    > unsigned value and the <-test could then be false


    Again being forced to correct myself: The other theoretical
    problem would be that a size_t value larger than UINT_MAX could be
    converted to an unsigned value smaller than the int-value. But that's
    still basic properties of C and about impossible to actually happen
    in a lexical analyser in general and in a lot of 'specific
    situations' where other constraints ensure that the actuall upper
    bound of the 'maximum' value is a lot smaller than 2^32 - 1.