Re: Testing strategy for compiler

Matthias-Christian Ott <ott@mirix.org>
Fri, 18 Jun 2010 10:45:21 +0200

          From comp.compilers

Related articles
Testing strategy for compiler kuangpma@gmail.com (kuangpma) (2010-06-16)
Re: Testing strategy for compiler gah@ugcs.caltech.edu (glen herrmannsfeldt) (2010-06-17)
Re: Testing strategy for compiler zaimoni@zaimoni.com (Kenneth 'Bessarion' Boyd) (2010-06-17)
Re: Testing strategy for compiler ott@mirix.org (Matthias-Christian Ott) (2010-06-18)
Re: Testing strategy for compiler gene.ressler@gmail.com (Gene) (2010-06-18)
Re: Testing strategy for compiler gneuner2@comcast.net (George Neuner) (2010-06-18)
Re: Testing strategy for compiler gah@ugcs.caltech.edu (glen herrmannsfeldt) (2010-06-18)
Re: Testing strategy for compiler ott@mirix.org (Matthias-Christian Ott) (2010-06-19)
Re: Testing strategy for compiler gah@ugcs.caltech.edu (glen herrmannsfeldt) (2010-06-19)
Re: Testing strategy for compiler jm@bourguet.org (Jean-Marc Bourguet) (2010-06-21)
[10 later articles]
| List of all articles for this month |

From: Matthias-Christian Ott <ott@mirix.org>
Newsgroups: comp.compilers
Date: Fri, 18 Jun 2010 10:45:21 +0200
Organization: Compilers Central
References: 10-06-037
Keywords: testing
Posted-Date: 18 Jun 2010 11:08:46 EDT

On Wed, Jun 16, 2010 at 07:18:20AM -0700, kuangpma wrote:
> Folks,
>
> Say I have written a hand crafted lexer/parser/code gen.... to make a
> complete compiler. The question is how to test it? Since user can have
> millions possible ways of writing their program (with many different
> types of syntax errors) and it is difficult to test all the possible
> cases. So is there any good ways to test the compiler? How do those
> big guys (MS/Borland...) tested their compiler? Thanks.


If you have a formal (generative) grammar for your language, you can
generate all valid strings of language and test whether your parser
recognises them. So you have practical tests which also cover obscure
programmes. You could use yagg [1] for this purpose.


With a formal description of your language (e.g. structural operational
semantics) you could use a theorem prover to test whether the generated
programmes are semantically correct. But I doubt that this process
can be fully automated and I think it might be not as useful with a
compiler since the compiler translates the programme and processor
executes it, so you can only test the result of the computation and
not the intermediate steps.


If you want to see realistic examples, you could have a look at Perl 5.
They have a big test suite with many regression tests.


Regards,
Matthias-Christian


[1] http://sourceforge.net/projects/yagg/
[Any useful language has an infinite number of valid strings. I suppose you could
generate enough to test that it can match each rule in the grammar. And this doesn't
address the harder question of verifying that it'll reject invalid programs. -John]



Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.