Flex and Dynamic Data buffer: Advanced technique

ymotiwala@yahoo.com (Yusuf Motiwala)
17 May 2002 00:25:31 -0400

          From comp.compilers

Related articles
Flex and Dynamic Data buffer: Advanced technique ymotiwala@yahoo.com (2002-05-17)
RE: Flex and Dynamic Data buffer: Advanced technique qjackson@shaw.ca (Quinn Tyler Jackson) (2002-05-23)
Flex and Dynamic Data buffer: Advanced technique cfc@world.std.com (Chris F Clark) (2002-05-23)
Re: Flex and Dynamic Data buffer: Advanced technique clint@0lsen.net (Clint Olsen) (2002-05-23)
| List of all articles for this month |

From: ymotiwala@yahoo.com (Yusuf Motiwala)
Newsgroups: comp.compilers
Date: 17 May 2002 00:25:31 -0400
Organization: http://groups.google.com/
Keywords: lex, question
Posted-Date: 17 May 2002 00:25:31 EDT

Hi,


This is more advanced problem than simply using yyinput, i need you help.


Is it possible to stop the flex when buffer exhausted and restart
again when more data is available. Flex should restart with previous
state where it left with previous buffer so that it can process
splitted tokens. For example,


Flex is parsing token "Hello"


Initally buffer has 3 byes "Hel", which does not match the token "Hello".
After reading 3 byes, buffer exhausts and hence flex stops.


Now after some time, flex receives more data. Now buffer content is "lo hi"
Flex should restart and be able to detect the token "Hello" which is splitted
between two buffers "Hel" and "lo hi".


I know I can do it with making one more process/thread which waits on pipe
or by other mechanism, unfortunately I can;t use it since this has to be
done by a single process. This process is doing other work also so it cant
simply be in loop to make flex waiting.


I will appreciate you help.


Thanks
Yusuf
[flex parsers expect the parser to call the lexer, and the lexer to call
the I/O, while you want to go the other way around. Either you need to
completely restructure the lexer, or else use a thread package. -John]



Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.