Reliability (was: Re: Optimizing Across && And ||)

"Ronald F. Guilmette" <>
Thu, 16 Mar 1995 18:46:33 GMT

          From comp.compilers

Related articles
Re: Optimizing Across && And || (1995-02-23)
Re: Optimizing Across && And || (1995-03-13)
Reliability (was: Re: Optimizing Across && And ||) (Ronald F. Guilmette) (1995-03-16)
| List of all articles for this month |

Newsgroups: comp.compilers
From: "Ronald F. Guilmette" <>
Keywords: optimize, design, testing
Organization: a2i network
References: 95-02-179 95-03-078
Date: Thu, 16 Mar 1995 18:46:33 GMT

David Keppel <> wrote:
>Opinion alert:
>... [Stuff about why implementors spend so much time worring about their
> SPECmark numbers deleted]...
>I think we're starting to see changes here already, but it's an
>uphill battle to sell reliability...

Tell me about it! I sell compiler test suites for C and C++. It's a hard
sell, even at the best of times, even though I price these things dirt
cheap (relative to what it would cost for the compiler vendors to reproduce
even a small fraction of my test cases).

I even had a guy at one local semiconductor firm say to me once "If we buy
your test suite, then we will find some more bugs in our compiler. So what?"

I kid you not! This is really what he said.

>... which is unpleasant to talk about...

Compiler quality is only unplesant to talk about for those folks that don't
have it. (I could name names here, but I won't.)

>... and hard to measure.

Rubbish! I've developed sophisticated automated techniques for producing
randomized compiler test cases that root out even the most subtle compile-
time bugs. Free samples are available for anonymous ftp in the files:

If you want to see your favorite C compiler screw up and give errors on
some perfectly valid C code (and maybe even give up the ghost) just try
compiling these examples. They found bugs in every one of the 20+ C
compilers I've tried then on so far (although some verdors have since
fixed the bugs which are triggered by my test cases).

Suggestion: If you try these test cases on some C compiler(s), I suggest
disabling all warnings when doing the compiles. Any actual _errors_ that
you get will be due to compiler bugs.

>Performance is less hard (but not easy) to measure and obviously desirable.

I can't remember who said it, but I am often reminded of a wonderful quote:
"It is easy to get incorrect answers infinitely fast."

-- Ron Guilmette, Sunnyvale, CA ---------- RG Consulting -------------------
---- E-mail: ----------- Purveyors of Compiler Test ----
-------------------------------------------- Suites and Bullet-Proof Shoes -

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.