Re: incremental compilation via shared library

>There exists a few other systems in the world working with a scheme similar
>to this.
>
>The main problems in such systems are the following:
>
>1. You has to use always the same environment for updating your source text.
> This seems to be a simple thing but restricts your freedom. It's more
> difficult to reach consistency. Note: there are a lot of people in love
> with 'vi'.
>
>2. The implementation of such systems is not so simple as people thinks.
> Especially in the case of any errors it's sometimes a little difficult
> to avoid inconsistent states.
>
>3. The most interesting thing in incremental compilers is the high
> compilation speed to avoid waiting time (== lost time). Some people say:
> the current hardware is so very fast that all such tricks are not necessary.
> I'm not convinced by this argument because I'm dayly waiting a lot of
> minutes for the results of compilation processes.
>
>4. There are many possibilities to increase the speed of compilers
> (see Lecture Notes of Computer Science 371). I have the impression that
> the compilers will be slower every day than faster. This is a little
> surprising. The available resources don't help in using very efficient
> data structures and algorithms.
>
>5. But I'm using C and develop a Fortran 90 compiler. Both languages are much
> more difficult as MODULA-2. C uses the terrible preprocessing. I think
> there are no possibilities to avoid this step if you want to develop
> an incremental C compiler.
>
> And Fortran 90 is a language with distributed declarations. It should be
> difficult to implement a truly incremental compiler (for declarations
> also).
>
>I'm not happy that from my experience I have to say that incremental
>compilers have a bad future. Sorry!
>
>I hope this will help to conclude the discussion about incremental
> compilation (or to animate?).


Both Digital Equipment Corporation, and Rational, supply Ada compilers that
support some kind of either Smart or Incremental compilation.


I work for Digital, and in particular worked on the implementation of Smart
Recompilation for our V3.0 release.


DIGITAL HAS BEEN SHIPPING A PRODUCT, DEC ADA V3.0 FOR OPENVMS VAX WITH THE
PROFESSIONAL DEVELOPMENT OPTION, FOR MOST OF CALENDAR YEAR 1993, THAT
IMPLEMENTS SMART RECOMPILATION. OUR USERS LOVE IT.




When we were surveying the area, we decided that the facts were


(a) Ada compilers on 1990's h/w are fast enough to negate the benefits of
        incrementation compilation, provided that you did Smart Recompilation.


(b) People will not put up with having to use a different editor, or
        development environment, to get the feature.




/Bevin




Smart Recompilation
-------------------


What is it?


    Background:


        Ada programs are made up of many source files containing the pieces of the
        program called 'compilation units'.


        Each compilation units depends on zero or more other compilation units,
        usually by using a language construct known as a 'with clause'. These
        dependencies create a natural compilation order where the depended on
        units must be compiled before those that 'with' them.


        Before linking the compilation system checks that all the units in the
        link are correct for each other. This check is usually implemented by
        simply verifying that, for each unit being linked, all the units it
        depended on are also being linked and that they are all the exact same
        version of these units that were compiled against, not some other unit
        with the same name.


        This same check is also applied to a compilation unit before another
        compilation unit is allowed to 'with' it.


    The problem:


        Compiling one of the earlier units again, perhaps because of a change to
        the source, or a desire to change the optimisation level or amount of
        debugging support, causes this method of implementing the check to fail.
        In order to link the application, all the 'obsolete' compilation units
        are going to have to be recompiled. This step may take a considerable
        amount of elapsed time - minutes, hours, even days in large applications.


    Smart Recompilation defined:


        A compilation system that implements Smart Recompilation modifies the
        check to be much more sophisticated. Instead of just checking for the
        exact same version of the depended on unit, a finer grained check is made
        that just the actual depended on 'fragments' of the depended on unit are
        the same.


        The compilation system still compiles the entire depended on unit
        [a process that even on the 10 mip machines of the early 90's,
        only takes a few seconds]. It still does full optimisation on
        this unit, and produces a single object file for it. However
        many of its dependents will not become obsolete, and even fewer
        of the dependents on the dependents. It is in not having to
        recompile these extra units that the major saving comes.


        So, the view presented to the outside world is simply that of a
        conventional compiler, except that it is instantanious in
        'recompiling' the 'obsolete' compilation units.


nb: The name 'Smart Recompilation' comes from a paper of the same
name, by Walter F. Tichy, in the July '86 edition of the
ACM Transactions on Programming Languages and Systems.


The technique has been implemented for C, Mary/2, and CHILL.
We believe that the DEC Ada V3.0 suite of compilers are the first
Ada compilers to have this technology.


    Smart Recompilation compared with Incremental Compilation:


        Another technique also exists for achieving this effect, known as
        incremental compilation. In this approach special editting tools,
        and a special compilation system including linkers, are used to treat
        compilation units as though they were made of many smaller independent
        fragments.


        This system will recompile all the fragments that depended on the changed
        fragment, all the fragments that depended on them, and so forth. It is
        possible that the system may recognise that there really wasn't a change
        to a fragment - an implementation of the idea of smart recompilation at
        the fragment rather than compilation unit boundary.


        Because this system uses many fragments, often thousands per
        unit, it places heavy demands on the development environment and
        may not support some of the capabilities of conventional systems,
        such as third party tools, and third party object libraries.






What size application/Who is it good for?


        EVERYONE!


        Whether the application being developed takes 5 minutes, 50 minutes, or 5
        or more hours to compile from scratch, it always has a few root
        compilation units that are depended on directly or indirectly by most of
        the other units.


        And yet people writing all small programs often wish to enhance such unit,
        and don't want to wait the 5 minutes before they can link their
        application; and people writing the larger size applications
        often have to resort to careful management and scheduling to try
        an avoid having the recompile slow down their development. And
        still put up with the inevitable delays when something slips through.






Demo'ing Smart Recompilation:


        Demo'ing any compiler is hard. Fortunately the effects of smart
        recompilation are readily seen, even with small programs.


        The major thing the customer is going to be concerned about is what class
        of changes to the sources do not result in major recompilations.




        Example 1. Changing Formatting, rearranging declarations, and adding
more variables, constants, and subprograms; and still being
able to relink without recompiling dependent modules.




$ ada/smart sys$input


package EG01_PKG is
        procedure P;
        procedure Q;
        V0 : String(1..3);
        C1 : constant String := "This is C1";
        C2 : constant String := "C2 C2 C2!!";
        V1 : String(4..20);
end;


package body EG01_PKG is
        procedure P is begin null; end;
        procedure Q is begin null; end;
end;


with EG01_PKG;
procedure EG01 is
begin
        if EG01_PKG.C1 /= "This is C1"
        or EG01_PKG.C2 /= "C2 C2 C2!!"
        then
raise Program_Error;
        end if;
end;


$ ada/smart sys$input


package EG01_PKG is
        V2 : String(1..100); -- ADDED
        V1 : String(4..20); -- REORDERED
        V0 : String(1..3);
        C3 : constant String := "Wasn't here before!"; -- ADDED
        C2 : constant String := "C2 C2 C2!!"; -- REORDERED
        C1 : constant String := "This is C1";
        procedure Q; -- REORDERED
        procedure P;
end;


$ acs link EG01
--

Return to the comp.compilers page.
Search the comp.compilers archives again.