Re: Unit testing a compiler

"Rafael R. Sevilla" <dido@imperium.ph> writes:


> I am currently writing a simple byte compiler for a simple dialect of
> Lisp, and am wondering what is the best practice for testing such a
> compiler. It is certainly possible to run the compiler on some test
> code and then compare the code it generates with some reference code,
> but as the code samples become more complicated that rapidly becomes
> unwieldy.


Right, that won't work. (Unless you're testing an assembler,
which is a different story.)


>...Would it instead be better to test the compiler by actually
> running the code it generates, and then comparing the results with what
> the code snippet is supposed to evaluate to?


Yes, that's the right answer.


The specification of a high-level language says what programs are
supposed to do, not what machine code is generated by the compiler.
So test what they do -- compare outputs of compiled programs against
expected outputs. Any difference is a regression worth investigating.


Changes (such as new optimizations) will likely change the machine
code, but if the output of compiled programs is unchanged, that's OK.


>...Any other ideas on how to
> go about testing the code generator?


Lots, but too much to fit in the margin of this post. ;-)


Here's just one: Every time you fix a bug, add a regression test to
make sure that particular bug doesn't rear its ugly head again.


OK, one more: Every time one of your customers shows you some source
code, turn it into a regression test for your compiler. Get as much
test code as you can get your hands on.


>...What about when optimizations are
> being performed?


Test with optimization on and off.


Above is about testing that optimization (etc) didn't break something.
That's hard enough, but testing that "optimizations" don't actually
pessimize is even harder.


- Bob

Return to the comp.compilers page.
Search the comp.compilers archives again.