Re: What should be check in Lexical Analyzer along with generating tokens?

Lex Spoon wrote:
>
> Here's a harder example:
>
> /* parse "this */ 10 /* and this" */
>
>
> It will scan as four tokens:
>
> /*
> parse
> "this */ 10 /* and this"
> */
>
>
> The parser will be in trouble now if it wants to pull out the "10".


If it hurts, don't do that ;-)


Seriously, IMHO the 10 should be considered part of the comment.
Otherwise, inserting /* */ around a piece of code will not reliably
comment it out.
Besides, most humans (well, at least myself *g*) will parse the initial
line as a single comment, why should the scanner use a different assumption?


> Thus, you seem to need a special scanner state to handle the
  > meat of a comment.


If you really insist on making the example into two comments: yes,
things will get ugly quickly.


Basically, it depends on the questions what's the contents of a comment:
is it a sequence of characters, or a sequence of tokens? That's
something that should be found in the language definition.


Regards,
Joachim

Return to the comp.compilers page.
Search the comp.compilers archives again.