Re: How to get at the entire input buffer string from inside lexer?

Eric Fowler <eric.fowler@gmail.com>
Mon, 30 Mar 2009 19:11:00 -0700

          From comp.compilers

Related articles
How to get at the entire input buffer string from inside lexer? eric.fowler@gmail.com (Eric Fowler) (2009-03-28)
Re: How to get at the entire input buffer string from inside lexer? DrDiettrich1@aol.com (Hans-Peter Diettrich) (2009-03-29)
Re: How to get at the entire input buffer string from inside lexer? m.helvensteijn@gmail.com (2009-03-29)
Re: How to get at the entire input buffer string from inside lexer? gah@ugcs.caltech.edu (glen herrmannsfeldt) (2009-03-29)
Re: How to get at the entire input buffer string from inside lexer? eric.fowler@gmail.com (Eric Fowler) (2009-03-30)
| List of all articles for this month |

From: Eric Fowler <eric.fowler@gmail.com>
Newsgroups: comp.compilers
Date: Mon, 30 Mar 2009 19:11:00 -0700
Organization: Compilers Central
References: 09-03-105 09-03-106
Keywords: lex
Posted-Date: 31 Mar 2009 14:26:52 EDT

I am past this, but since you asked:


I am dealing with a data format that comprises strings of the form:
[leader_string]<tail_string><crlf>




Where the optional leader_string looks like:
<delimiter><variable_data><checksum><delimiter>


And the required tail_string looks like:
<header><variable_data_depending_on_header><checksum>


So there are two different kinds of embedded checksum: one is a check
on the optional leading sequence, the other is on the required
trailing sentence. There is no checksum for the entire line. Since I
am already committed to parsing one line at a time (which is what I
get from client programs), I wanted to leverage the intelligence in
the lexer to spot the delimiters and pick out the checksum sequences
for processing. I could just pre-process the entire string as soon as
I see it, scanning for checksums as I go, but then I have to map the
recovered checksums to the fragments identified by the lexer and
parser, which does me no good in the long run. So the problem as I got
it was, "when the lexer sees a delimiter for beginning or end of a
fragment, how do I know what the original string looked like between
those points?". And I came up with an answer I can live with.


Thanks to all as usual.


Eric


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.