Re: errata for the Dragon Book?

marc@oscar.noc.cv.net (Marc Spitzer) writes:


> Andrew Luke NESBIT wrote:
...
> >Anyway, here is the suspected typo from page 87, line 8, just under the
> >heading "Attributes for Tokens":
> >
> > "When more than one pattern matches a lexeme, the lexical analyzer
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > must provide additional information about the particular lexeme
> > that matched to the subsequent phases of the compiler."
> >
> >should read:
> >
> > "When a pattern matches more than one lexeme..."
>
> I don't agree, those are 2 different sentences. The first one says:
> given 1 lexeme and 2 or more patterens that match the lexeme you need
> to have more information to find the correct pattern to apply.
> The second one says:
> I have more then 1 lexeme that matches the same patteren, this cannot
> happen in lex. It only deals with 1 token at a time, so you could not
> have it match multple lexemes into the same patteren at the same time.
> You are going left to right on this and you cannot have 2 lexemes in
> the same part of your input buffer.
>
> marc


I don't think you read the full quote from the dragon book closely
enough, or looked carefully enough at the context. Note first of all
the the quote's subject is "the lexical analyzer" and the sentence
ends with "to the subsequent phases of the compiler." So we are not
talking about additional information provided from some external
source (e.g., the human author) to the lexical analyzer, but rather
about additional information provided from the lexical analyzer to the
later parts of the compiler. In particular, if you look at the
context that wasn't quoted, the discussion concerns attributes. (For
example, the attribute on the token INTEGER tells whether the specific
lexeme matching the pattern was "314" or "271".)


This also explains where you go wrong when you say that this
phenomenon (two lexemes match one pattern) can't happen in lex. Yes,
only one lexeme matched the pattern at this point in the input; but
the later parts of the compiler need to know which one lexeme did so.
It *could* have been others; the token type isn't specific enough.
Hence the attribute. That is all this quote is trying to say, and the
suggested correction is correct.


  -Max Hailperin
    Associate Professor of Computer Science
    Gustavus Adolphus College
    800 W. College Ave.
    St. Peter, MN 56082
    USA
    http://www.gustavus.edu/~max/

Return to the comp.compilers page.
Search the comp.compilers archives again.