yyless and pattern matching

Hello,


I've found a little peculiarity in flex I don't understand. Perhaps
you can explain this to me? I beleive I have isolated the issue in the
appended example.


In the example, comments('#' signs) are collected using yymore for
returning as one token. To detect the end of a series of comments the
lexer looks ahead for a char that is not a '#' at which point it calls
yyless to put that one char back. Now however if you try to match that
char with the '^' beginning of line op it fails. It's as though
because the character was matched previously, the regular expression
state required to match the char put back at the beginning of a line
is not valid anymore.


Here's the example lexer, input, and sample run to illustrate. I am
interested in how this problem is dealt with and moreover how flex works
exactly. Bare with me, I have no formal training and learned all of what
I know from a few hours with the O'Reilly book on lex and yacc.


Thanks,
Michael B. Allen
PS: Thanks for your help on my previous post Robert and John.


--LEXER-------------


%x X
%%


#\n {
        printf( "first_comment{%s}", yytext );
        yymore();
        BEGIN X;
      }
^t {
        printf( "BEGAN WITH 't'!\n" );
      }
<X>#\n {
        printf( "more_comment{%s}", yytext );
        yymore();
      }
<X>[^#] {
        yyless( yyleng - 1 );
        printf( "no_longer_comment{%s}", yytext );
        BEGIN INITIAL;
      }


%%


--INPUT-------------


#
#
#
t
t


--TEST_RUN----------


[miallen@angus parsers]$ test.parser < input
first_comment{#
}more_comment{#
#
}more_comment{#
#
#
}no_longer_comment{#
#
#
}t << 't' is at the beg of line but didn't match?
BEGAN WITH 't'! << now it did

Return to the comp.compilers page.
Search the comp.compilers archives again.