Re: Intricate problem with scannerless LALR(1) parser

> However, that keyword-feature has one side effect which I would
> discuss on the mailing list. Given the following grammar:
>
> start: a
> a: b 'XX'
> b: c | '[' b ']'
> c: 'X' | c 'X'
>
> [Your grammar is ambiguous. To see where, replace "XX" with xx and
> define it like this: xx: "X" "X"
>
> Assuming you want to use normal tokenizing rules, your "X" token is
> really "X followed by something other than a letter or digit, and if
> the something is white space, skip over the white space. Oh, and skip
> comments, too." Now you know why we use separate lexer and parser
> generators, because they need separate state machines to keep the
> parser grammar frome exploding. -John]


I agree with John.


XX is valid, but ambiguous, either a keyword or two X's.


A normal compiler front end would determine that XX is a keyword first
by doing a keyword lookup, before passing the token (XX) to the parser.


This would require a separate lexer that creates tokens to be looked up
before parsing them.


This is good example, showing one of the disadvantages of scannerless
parsing systems.


Languages like PL/I which have no reserved words would probably be
a nightmare for scannerless parsing systems.




Paul B Mann
http://lrgen.com

Return to the comp.compilers page.
Search the comp.compilers archives again.