Re: Testing strategy for compiler

On Jun 16, 9:18 am, kuangpma <kuang...@gmail.com> wrote:
> Folks,
>
> Say I have written a hand crafted lexer/parser/code gen.... to make a
> complete compiler.


Basically where I am with my own (Z.C++), yes.


> The question is how to test it? .... So is there any good ways to test the
compiler?


I ended up studying DejaGNU and the related POSIX standards to get a
sense of what would be needed out of a test driver system.


Z.C++ has three layers of testing:
* static assertions that prevent compilation if program-wide constants
are wrong relative to how they're used.
* assertions that both partially implement Design By Contract, and
check whether program-wide constants are wrong relative to how they're
used (these checks are run-time because they can't be compile-time).
* A test driver script that examines whether test cases work (or don't
work) as expected. The main reason I don't use DejaGNU is that
there's no easy way to build Expect (which DejaGNU relies on) as a
MingW32 binary; it relies on *NIX fork. I can't rule out
transitioning to lit from shell scripts for the test driver at some
point.


If it's not clear, from the parser design, how to attain 100% code
coverage of the parser, the test suite isn't even trying to be
complete.


> How do those big guys (MS/Borland...) tested their compiler? Thanks.


GCC and Clang both mostly rely on a test driver setup that feeds test
cases into the compiler and checks whether the test cases succeed or
fail as expected. GCC uses DejaGNU; Clang has used DejaGNU for some
time but appears to be trying to migrate away to an internal test
driver system shared with LLVM, lit .


Clang also has occasional extended testing using randomly generated
programs.


[All of the above is public information.]
[If it wasn't public before, it sure is now. -John]

Return to the comp.compilers page.
Search the comp.compilers archives again.