Re: Flex question

bliss@sp64.csrd.uiuc.edu (Brian Bliss)
Mon, 3 Aug 1992 23:03:55 GMT

          From comp.compilers

Related articles
Flex question suzi@linus.mitre.org (1992-07-31)
Re: Flex question bliss@sp64.csrd.uiuc.edu (1992-08-03)
Flex question guidomi@cogs.susx.ac.uk (Guido Minnen) (1999-04-09)
| List of all articles for this month |

Newsgroups: comp.compilers
From: bliss@sp64.csrd.uiuc.edu (Brian Bliss)
Organization: UIUC Center for Supercomputing Research and Development
Date: Mon, 3 Aug 1992 23:03:55 GMT
References: 92-07-117
Keywords: flex, performance

suzi@linus.mitre.org writes:
|> I have been trying to determine what it is in flex rules that causes the
|> yy_nxt and yy_chk tables to increase in size, but am not getting anywhere.


Use different start states when parsing normal program text, comments,
and strings. i.e., say


----------------------


%START NORMAL
%START COMMENT
%START STRING


ws [ \t]


%%


<NORMAL>\" {
                                                                                      BEGIN STRING;
                                                                                }
<STRING>\" {
                                                                                      <string handling stuff>
                                                                                      BEGIN NORMAL;
                                                                                      return (T_STRING);
<STRING>[^"\n\\]+ {
                                                                                      yymore ();
                                                                                }
<NORMAL>keyword {
                                                                                        return (T_KEYWORD);
                                                                                }


%%


void lex_init () { BEGIN NORMAL; }




-----------------


and call lex_init() before you call yylex() to make the lexer start out in
the NORMAL state. Recognizing keywords and "normal" program text in the
default lex state will often result in a 10x increase in the size of the
lexer for a C/pascal/what-have-you language (and chances are that you will
code in a bug where buffers overflow on comments, or keywords/strings are
incorrectly recognized within a comment, if you fail to use the start
state approach).


bb
--


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.