Re: Mysterious triple test failures

Kaz Kylheku <kaz@kylheku.com>
Sat, 3 May 2014 22:19:06 +0000 (UTC)

          From comp.compilers

Related articles
Mysterious triple test failures nmh@t3x.org (Nils M Holm) (2014-05-03)
Re: Mysterious triple test failures kaz@kylheku.com (Kaz Kylheku) (2014-05-03)
Re: Mysterious triple test failures mrs@Kithrup.COM (2014-05-04)
Re: Mysterious triple test failures nmh@t3x.org (Nils M Holm) (2014-05-04)
Tripe test failure update nmh@t3x.org (Nils M Holm) (2014-05-04)
Re: Mysterious triple test failures alexfrunews@gmail.com (2014-05-04)
Re: Triple test failure update cdodd@acm.org (Chris Dodd) (2014-05-23)
Re: Triple test failure update nmh@t3x.org (Nils M Holm) (2014-05-23)
[1 later articles]
| List of all articles for this month |

From: Kaz Kylheku <kaz@kylheku.com>
Newsgroups: comp.compilers
Date: Sat, 3 May 2014 22:19:06 +0000 (UTC)
Organization: Aioe.org NNTP Server
References: 14-05-003
Keywords: debug
Posted-Date: 03 May 2014 19:18:54 EDT

On 2014-05-03, Nils M Holm <nmh@t3x.org> wrote:
> In the recent months, several people who ported and/or used my
> SubC compiler[1] reported that the triple test failed, but the
> compiler still worked fine.


If you don't understand it, who can? Haha.


This isn't real-time-event-processing software with nonrepeatable
inputs and scheduling patterns. :)




>
> I am running the triple test as follows:
>
> Subc sources ---[ third-party C compiler ]---> stage-0 SubC
> Subc sources ---[ stage-0 SubC ]---> stage-1 SubC
> Subc sources ---[ stage-1 SubC ]---> stage-2 SubC
>
> (where x---[y]--->z means "compile x with y to z"). Finally the
> stage-1 and stage-2 compilers are compared. When they are equal,
> the triple test is passed.
>
> Now it has happened that this test failed, resulting in stage-1
> and stage-2 compiler of identical size, but with tens of thousands
> of differences.


It means you have some uninitialized variable or something which leads
the third-party-C-compiled SubC to write different code from the
SubC-compiled SubC.


> However, both compilers seem to work fine.


Do these tests execute with zero errors under Valgrind on a Linux box?


> Further tests have shown that the assembly language output of
> the differing compilers is identical.


But that of stage-0 and stage-1 isn't identical: they produce
different output when compiling SubC. Necessarily so: the stages use
completely different compilers. But those outputs, in turn, have a different
behavior: they produce different code!


If you were to iterate and make a stage-3 with stage-2, it would probably come
out identical to stage-2. I.e. there is a "fixed point" when you feed
the compiler into itself.


> This happened on 386-based OpenBSD and x86-64-based Linux systems.
>
> So, what is going on here? Does anybody have an explanation?
>
> And: to work around the problem, I thought about comparing the
> assembly output of the compilers instead of the binaries. This
> should yield the same results, shouldn't it?


Only if the assembly stage is responsible for the differences.


> Please let me know what you think!


Disable all special code-generation options like optimizations.
If that makes the problem go away, re-introduce them one by one until
a difference shows up.


A bug could be causing some optimization or other code transformation decision
to be done differently. If such a decision is semantics-preserving
(which such decisions often are) then the differing variants of the
translated code behave the same way.


It still points to a bug, which is why it's smart to do that comparison between
the stages.


You have a simple case here of a program compiled by two different C compilers
producing different output. How does that happen? I mean, let me count
the ways, right? Dependencies on orders of evaluation, uninitialized
storage, or numerous other "undefined behaviors". This is why a sweep with
a debugging tool like Valgrind wouldn't hurt.


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.