Re: 16 bits lexers...

Scott Stanchfield <>
8 May 1997 21:26:50 -0400

          From comp.compilers

Related articles
16 bits lexers... (Darius Blasband) (1997-05-04)
Re: 16 bits lexers... (Scott Stanchfield) (1997-05-08)
Re: 16 bits lexers... (Jerry Leichter) (1997-05-08)
| List of all articles for this month |

From: Scott Stanchfield <>
Newsgroups: comp.compilers
Date: 8 May 1997 21:26:50 -0400
Organization: MetaWare Incorporated
References: 97-05-041
Keywords: lex, i18n

Darius Blasband wrote:
> Hello,
> There have been quite some arguments about how to implement scanners
> dealing with 16 bits (unicode) transitions. One of the major issues

Another approach is to do what Terence Parr has done for PCCTS 2.0 --
the scanner is recursive-descent, just like the parser. No tables.
It will eventually have some nice auto-inlining optimizations, and the
generated code is actually pretty similar to what you might write by
hand for a scanner. It's written in Java and handles Unicode...

Take a peek at
for details.

Scott Stanchfield
MetaWare Incorporated
Santa Cruz, CA

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.