Re: Fast code vs. fast compile

In comp.compilers, dennis@netcom.com (Dennis Yelle) writes:
| As a compiler user, I am (almost) always more concerned about fast
| code rather than fast compilation. My makefiles generally build code
| with max optimization and I can remember only about a dozen times in
| my LIFE that I got myself so confused that I needed to do so many
| recompiles that I changed the makefile to make the compile go faster.


Speaking as another compiler user, compilation time can definitely be
a critical path in the development process. One project I was working
on about 7 years ago was about (er, exactly) 285543 lines of code in
approximately (er, exactly) 1342 files. On the hardware we had at the
time a complete recompile was an 8 hour job. It was so bad, I played
with the makefile and some shell scripts and distributed the build
over 7 machines to speed things up.


So the question is, why do large projects need to be completely
recompiled so often? Theoretically, make and/or precompiled headers
should mean only a few things need to be recompiled at any given time.
I can imagine (er, remember) some external factors that occasionally
force a rebuild:


      * compiler update or switch to new vendor
      * OS updates
      * file system corruption
          - running make on NFS has problems when server and client
              clocks are a few minutes off
          - restore from incremental backups throws doubt
              on the file modification dates


But most often, the rebuild is internally motivated:


      * somebody changed "common.h" or its equivalent
      * change of "mode" e.g. boss says program is too slow
          so we recompile with profiling on; next day boss says it
          crashed in front of client so we need debugging
          on now
      * mysterious bugs that make people want to recompile
          everything just for peace of mind (the scary part
          is that this fixed the problem about 1/3 the time, and
          yes I know this is a fallacy so don't email about it)
      * porting


And now, my point: the larger the project, the more often these events
happen and the greater the consequences. I could easily see a
million-line project with 10 or 20 people ending up needing a rebuild
every day. The timing of rebuilds ends up being a gigantic headache,
one that keeps somebody (er, me) optimizing makefiles when I should be
optimizing the program the makefile is supposed to be building. As
hardware improves, the size of project at which compilation time
becomes an issue recedes into the distance, but I'd say it'll be
another 10 years before it becomes a non-issue.


Two other points:


      * I will not trade compilation speed for correctness of compiled
          code. Buggy output from the compiler is 1000 times worse a sin
          than merely being slow.
      * An often neglected contributor is linker effeciency. We were
          seeing 1/2 hour link times before we gave up and switched to shared
          libraries, even though we didn't really need them otherwise.)


Back to work.


            mikey
[I found that 600K lines with 10 people plenty to require nightly
rebuilds. Half the time they didn't work, due to interactions among
roughly simultaneous updates. -John]


--

Return to the comp.compilers page.
Search the comp.compilers archives again.