NULs in flex scanner input

Hi,


I'm building a parser for the Procedure Language of Wang VS
systems. Problem is that one of the scripts I received as test inputs
contains embedded NUL characters. Since my string rules apparently
weren't matching them, I changed the lexer to consume the string char
per char as soon as a quote was seen. However, the problem remains;
input() will return a single NUL, and after that it first returns
parts of the previous line, and then goes on to read the rest of the
files (resulting in an unterminated string, as far as flex is
concerned). I tried redefining YY_END_OF_BUFFER_CHAR to prevent flex
from thinking it had hit EOB, but this doesn't work (flex just prints
'fatal flex scanner internal error - end of buffer missed').


Can anyone give me any pointers on how to accept a series of NULs as
valid input? Thanks in advance.


PS: I'm not 100% certain the NULs actually need to be there; it is
possible that spaces would do just as well - but since this is a
procedure that's been in use for several years, I'm inclined to adjust
the lexer rather than the input file. Especially since the parser will
be part of a conversion tool, and will likely have to deal with a
number of dubious input files.


Tim Van Holder
Falcon Software NV

Return to the comp.compilers page.
Search the comp.compilers archives again.