Context sensitive scanner ?

Hello all,


We are busy writing a language where the following constructs occur:


1) Literal reals (such as 1.2),
2) Nested index operations on arrays (such as x.1.2).


Obviously, indices are always unsigned positive integers.


Since the scanner is not context-sensitive, it does not understand that
the second expression should be returned as IDEN x, DOT, NUM 1, DOT NUM 2


I am aware of the capability of creating states in Lex, but I do not
really like it, since it heavily depends on the exact moment of executing
when YACC statements (at the end of a rule).


I'd rather want to know what tokens the parser is expecting, and recognize
only those tokens (maybe with an exception on keywords) in the scanner.


I have never heard of such a tool, so my questions are:
a) Does this exist, or I am trying something completely new here ?
b) Are there tools (preferably compatible with Lex/Yacc grammar syntax) ?


Albert
---
[Well, disregarding the question of whether it's a good idea to write
languages with lexical puns that practically beg people to write code
that the compiler will misinterpret, I'd have the lexer return
integers and dots as separate tokens and put reals together in the
parser. -John]
  --

Return to the comp.compilers page.
Search the comp.compilers archives again.