Re: Compiler stress tests? (Clyde Smith-Stubbs)
4 Jan 1997 20:49:22 -0500

          From comp.compilers

Related articles
Compiler stress tests? (1997-01-02)
Re: Compiler stress tests? (Duane Sand) (1997-01-03)
Re: Compiler stress tests? (1997-01-03)
Re: Compiler stress tests? (1997-01-04)
Re: Compiler stress tests? (1997-01-12)
Re: Compiler stress tests? (1997-01-16)
Re: Compiler stress tests? (Stephen Baynes) (1997-01-17)
Re: Compiler stress tests? (John Haxby) (1997-01-22)
| List of all articles for this month |

From: (Clyde Smith-Stubbs)
Newsgroups: comp.compilers
Date: 4 Jan 1997 20:49:22 -0500
Organization: Compilers Central
References: 97-01-020 97-01-024
Keywords: testing

Cliff Click wrote:

> In short, we throw the kitchen sink at the compiler.
> Beta testers still find bugs with amazing ease.

There's a good reason for this. Simply throwing a large test suite
at a compiler isn't enough - you also have to have some way of
verifying that you have actually exercised the entire compiler.
Ideally you would verify that every possible line of code the compiler
could produce has been produced during a test suite run. In practice
this is pretty hard, but you can come close.

I don't believe the test suite vendors can help you with this -
it's very specific to the compiler internals.

It's quite frightening when you apply this kind of verification
to an existing compiler - we find it's normal that the test suite
has only been testing about 70% of the compiler! Of the balance,
some turns out to be unreachable, much proves to be correct, but
some is bound to be wrong.

The time spend on conducting this kind of thing is considerable -
but is repaid in spades in reduced tech support calls.
  Clyde Smith-Stubbs | HI-TECH Software, | Voice: +61 7 3354 2411 | P.O. Box 103, Alderley, | Fax: +61 7 3354 2422 | QLD, 4051, AUSTRALIA. |

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.