[9fans] Replacements for lex - Plan9

This is a discussion on [9fans] Replacements for lex - Plan9 ; Has anyone had success with lexer-generators other than lex under Plan 9? I'm taking a compilers class this semester and I'd like to do as much work under Plan 9 as possible. I looked around online and found re2c, which ...

+ Reply to Thread
Results 1 to 13 of 13

Thread: [9fans] Replacements for lex

  1. [9fans] Replacements for lex

    Has anyone had success with lexer-generators other than lex under Plan
    9? I'm taking a compilers class this semester and I'd like to do as
    much work under Plan 9 as possible.

    I looked around online and found re2c, which looked interesting except
    it's written in C++.

    For languages sufficiently like C, is regexp(2) suitable?

    --Joel

  2. Re: [9fans] Replacements for lex

    On Fri, Jan 19, 2007 at 01:58:06PM -0500, Joel Salomon wrote:
    > Has anyone had success with lexer-generators other than lex under Plan
    > 9? I'm taking a compilers class this semester and I'd like to do as
    > much work under Plan 9 as possible.
    >
    > I looked around online and found re2c, which looked interesting except
    > it's written in C++.
    >
    > For languages sufficiently like C, is regexp(2) suitable?


    I think that most people roll their own lexical analyzers under Plan 9.
    That's typically not too hard to do, though.

    - Dan C.


  3. Re: [9fans] Replacements for lex

    My suggestion is just to write scanners by hand. I've written dozens
    and they take only a few minutes. While a half hour seems long
    compared to the speed of an automaticly generated scanner, the time
    difference over the total project is quite small. And besides. You
    want to learn how to write compilers.


    > Has anyone had success with lexer-generators other than lex under Plan
    > 9? I'm taking a compilers class this semester and I'd like to do as
    > much work under Plan 9 as possible.
    >
    > I looked around online and found re2c, which looked interesting except
    > it's written in C++.
    >
    > For languages sufficiently like C, is regexp(2) suitable?
    >
    > --Joel



  4. Re: [9fans] Replacements for lex

    take a look at the lexing in the c compiler -- /sys/src/cmd/cc/lex.c.

    - erik

  5. Re: [9fans] Replacements for lex

    yeah, great example

    On 1/19/07, erik quanstrom wrote:
    > take a look at the lexing in the c compiler -- /sys/src/cmd/cc/lex.c.
    >
    > - erik
    >



    --
    Federico G. Benavento

  6. Re: [9fans] Replacements for lex

    > I think that most people roll their own lexical analyzers under Plan 9.
    > That's typically not too hard to do, though.


    That's likely what I'll do for the final project (probably yet another
    C complier, but I might try my hand with [a subset of] D), but the
    professor has said he'll want us to learn to use
    mechanically-generated scanners and compare the results.

    I expect the lexer in /sys/src/cmd/cc/ will be a good example.

    --Joel

  7. Re: [9fans] Replacements for lex

    >I expect the lexer in /sys/src/cmd/cc/ will be a good example.

    it's a little more elaborate than some (not gcc!) because it takes on limited C preprocessing
    duties as well. you might also observe that the author enjoys his gotos, but since it is essentially
    a finite state machine that seems fair enough, and furthermore he's allowed to take advantage of his
    `licence to GOTO' awarded after a killer early paper. (Since that was about automata, it should probably
    be an Earley paper, except that it's slightly earlier than that.) anyway, he meant no harm.

  8. Re: [9fans] Replacements for lex

    i think the ken c lexer is a very good example. it shows how
    goto, well applied, can be a very useful tool for an ad hoc problem.
    it also shows that in practice lexers and parsers are seldom as
    separate as they appear in textbooks.

    on the other hand, rc has a very hairy lexer. hoc has a small, elegant one.
    i believe the _practice of programming_ goes into detail about it.

    - erik

    On Fri Jan 19 15:37:42 EST 2007, forsyth@vitanuova.com wrote:
    > >I expect the lexer in /sys/src/cmd/cc/ will be a good example.

    >
    > it's a little more elaborate than some (not gcc!) because it takes on limited C preprocessing
    > duties as well. you might also observe that the author enjoys his gotos, but since it is essentially
    > a finite state machine that seems fair enough, and furthermore he's allowed to take advantage of his
    > `licence to GOTO' awarded after a killer early paper. (Since that was about automata, it should probably
    > be an Earley paper, except that it's slightly earlier than that.) anyway, he meant no harm.


  9. Re: [9fans] Replacements for lex

    > you might also observe that the author enjoys his gotos

    If you want to go somewhere, goto is the best way to get there. - ken



  10. Re: [9fans] Replacements for lex

    >> you might also observe that the author enjoys his gotos
    >
    > If you want to go somewhere, goto is the best way to get there. - ken
    >
    >


    Goto Statements Considered Harmful
    -- Niklaus Wirth

    (Dijkstra wrote the letter, but Wirth gave it its title.)

    There's a really good scanner in Wirth's compiler book.

    http://www.oberon.ethz.ch/WirthPubl/CBEAll.pdf


  11. Re: [9fans] Replacements for lex

    On Fri, Jan 19, 2007 at 01:58:06PM -0500, Joel Salomon wrote:
    > For languages sufficiently like C, is regexp(2) suitable?


    For something that simple, you might as well just hand write it,
    particularly if you're going to have to support somethig typedef-like.

  12. Re: [9fans] Replacements for lex

    On Fri, Jan 19, 2007 at 02:26:47PM -0500, Joel Salomon wrote:
    > >I think that most people roll their own lexical analyzers under Plan 9.
    > >That's typically not too hard to do, though.

    >
    > That's likely what I'll do for the final project (probably yet another
    > C complier, but I might try my hand with [a subset of] D), but the


    You'll learn a lot more if you don't try anything like C
    let alone C++ or D. For a semester-length project, you'll
    learn far more if you work on a much smaller language --
    ideally domain specific, in my opinion -- and really see it
    through in detail.

  13. Re: [9fans] Replacements for lex

    I taught a course in advanced compiler construction where the
    students were required to write a lisp interpreter in C and then
    write a rudimentry C compiler in their lisp.

    The "bits you don't have to do" included the c lex function which
    was presented to them in lisp as a (c:token) function. I've written
    a lot of lex-ers and it was just the same code again (in c) which
    stopped the students from spending all of their time trying to do
    the same in lisp. This enabled them to concentrate on the more
    interesting tasks (I also supplied the lisp alloc and gc code).

    It was a fun course and I hope things were learnt.

    For most stuff something like lex is fine. Have a try at getting
    reg-exp useable and I'll type in a lex-er
    faster than you can debug the reg-exps (a chore i decided was
    a waste of time for the course).

    brucee

    On 1/20/07, William Josephson wrote:
    > On Fri, Jan 19, 2007 at 02:26:47PM -0500, Joel Salomon wrote:
    > > >I think that most people roll their own lexical analyzers under Plan 9.
    > > >That's typically not too hard to do, though.

    > >
    > > That's likely what I'll do for the final project (probably yet another
    > > C complier, but I might try my hand with [a subset of] D), but the

    >
    > You'll learn a lot more if you don't try anything like C
    > let alone C++ or D. For a semester-length project, you'll
    > learn far more if you work on a much smaller language --
    > ideally domain specific, in my opinion -- and really see it
    > through in detail.
    >


+ Reply to Thread