Re: Testing strategy for compiler

Matthias-Christian Ott <ott@mirix.org> wrote:
> On Wed, Jun 16, 2010 at 07:18:20AM -0700, kuangpma wrote:


>> Say I have written a hand crafted lexer/parser/code gen.... to make a
>> complete compiler. The question is how to test it? Since user can have
>> millions possible ways of writing their program
(snip)


> But I doubt that this process can be fully automated and I
> think it might be not as useful with a compiler since the
> compiler translates the programme and processor executes it,


(snip)


> [Any useful language has an infinite number of valid strings.


Yes, more than the million indicated above.


> I suppose you could generate enough to test that it can match
> each rule in the grammar.


As I previously noted about Knuth's tests for TeX and Metafont, his
goal is to make sure that the test executes every part of the program,
except for that processing fatal errors.


Now, that makes more sense for interpreters like TeX and Metafont than
it does for compilers, though one could require, similarly, that every
part of the compiler program be tested. (and for table-driven code
that every table entry be used.)


> And this doesn't address the harder question of verifying that it'll
> reject invalid programs. -John]


Except for fatal errors, you could test one wrong example of each
rule. That doesn't protect against interactions between rules, but
does help.


Well, the TeX tests, as distributed, make more sense as regression
tests. That is, to verify that a modified implementation still
performs as it is supposed to. (Where modified includes compiling the
same source on a different system.)


In statistical terms, it protects against random errors but not
systematic errors.


-- glen

Return to the comp.compilers page.
Search the comp.compilers archives again.