Re: Buffered input for a lexer?

"Randall Hyde" <>
23 Apr 2002 23:48:06 -0400

          From comp.compilers

Related articles
[17 earlier articles]
Re: Buffered input for a lexer? (2002-04-17)
Re: Buffered input for a lexer? (Randall Hyde) (2002-04-19)
Re: Buffered input for a lexer? monnier+comp.compilers/news/ (Stefan Monnier) (2002-04-19)
Re: Buffered input for a lexer? (Randall Hyde) (2002-04-20)
Re: Buffered input for a lexer? (Joachim Durchholz) (2002-04-23)
Re: Buffered input for a lexer? (Ray Dillinger) (2002-04-23)
Re: Buffered input for a lexer? (Randall Hyde) (2002-04-23)
| List of all articles for this month |

From: "Randall Hyde" <>
Newsgroups: comp.compilers
Date: 23 Apr 2002 23:48:06 -0400
Organization: Prodigy Internet
References: 02-04-061 02-04-081 02-04-097 02-04-116 02-04-117 02-04-121 02-04-131
Keywords: lex, performance, comment
Posted-Date: 23 Apr 2002 23:48:06 EDT

Joachim Durchholz wrote in message 02-04-131...
>Actually my language is (will be) much more high-level than assembly,
>so I either have to spend a lot of time with optimization, or be
>content with a slow executable. In both cases, lexer speed is one of
>my lest concerns. Which leads me to the question what's "fast enough"
>for an assembler. Is 10kLOC/s enough? Do you need more? Is less
>acceptable - and if so, *how much* less? How much of an improvement is
>the speed really worth?

Well, the Lisa 816 assembler running on a 2.5 MHz Apple II gs was
moving along at about 120,000 lines/min (20kLOC/s). I'd hope that
today's machines, being roughly 1,000 times faster (or better) could
do a whole lot better.

For me, instantaneous is fast enough :-).
The current HLA lexer is a pig. I'm really embarassed to mention
how I handle macros and the like (I "unget" the characters one at
a time - later versions of Flex than I started with allow you to switch
input to a string, but I've been afraid to change the code to take
advantage of this because it's probably fragile).

Definitely, though optimization in a HLL is a bigger concern than
scanning. However, your users can always turn optimization off
(I'm assuming you'll provide that option) to get fast compile times.
Then the lexer becomes the bottleneck.
Randy Hyde
[I gather that the lexer is often the slowest part of modern compilers,
because it's the only phase that has to look at each character of the
input individually. But this thread is still over. -John]

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.