Re: what scanner scheme is efficient?

John Lilley (jlilley@empathy.com) wrote:
: What sample have you examined to arrive at the conslusion that *most*
: scanners require backing-up? The languages that I know of (C, C++,
: Pascal) do not require any backtracking to support their keyword sets.


For some reason John's posting never made it to our site. He also sent
it to me via private email, so that was how I answered it. Anywhere,
here's my reply.


- Vern


A Johnstone <adrian@dcs.rhbnc.ac.uk> wrote:


> What sample have you examined to arrive at the conslusion that
> *most* scanners require backing-up? ...
> Most of the time, a single character of lookahead with no
> backtracking is sufficient.


Pick a flex scanner you have lying around and run it through flex -b and
you'll see what I mean.


You're right that keywords usually aren't a problem because of generic
identifier rules. I picked them as an example because they show how
backing up works clearly. The sorts of rules that cause problems are
string constants, escape sequences, floating point representations, and
multi-character operators.


                                Vern
--

Return to the comp.compilers page.
Search the comp.compilers archives again.