Re: Baby X is bor nagain

On Mon, 24 Jun 2024 15:00:26 +0100
bart <bc@freeuk.com> wrote:

On 24/06/2024 14:09, Michael S wrote:

On Fri, 21 Jun 2024 22:47:46 +0100
bart <bc@freeuk.com> wrote:
 

On 21/06/2024 14:34, David Brown wrote: 

On 21/06/2024 12:42, bart wrote: 

On 21/06/2024 10:46, David Brown wrote: 

>
>
I understand your viewpoint and motivation.  But my own
experience is mostly different.
>
First, to get it out of the way, there's the speed of
compilation. While heavy optimisation (-O3) can take noticeably
longer, I never see -O0 as being in any noticeable way faster
for compilation than -O1 or even -O2. 

>
Absolute time or relative? 

>
Both.
    

For me, optimised options with gcc always take longer: 

>
Of course.  But I said it was not noticeable - it does not make
enough difference in speed for it to be worth choosing.
    

   
   C:\c>tm gcc bignum.c -shared -s -obignum.dll        # from
cold TM: 3.85 

>
Cold build times are irrelevant to development - when you are
working on a project, all the source files and all your compiler
files are in the PC's cache.
>
    

   
   C:\c>tm gcc bignum.c -shared -s -obignum.dll 
   TM: 0.31
   
   C:\c>tm gcc bignum.c -shared -s -obignum.dll -O2 
   TM: 0.83
   
   C:\c>tm gcc bignum.c -shared -s -obignum.dll -O3 
   TM: 0.93
   
   C:\c>dir bignum.dll 
   21/06/2024  11:14            35,840 bignum.dll 

>
Any build time under a second is as good as instant.
>
I tested on a real project, not a single file.  It has 158 C files
and about 220 header files.  And I ran it on my old PC, without
any "tricks" that you dislike so much, doing full clean
re-builds.  The files are actually all compiled twice, building
two variants of the binary.
>
With -O2, it took 34.3 seconds to build.  With -O1, it took 33.4
seconds.  With -O0, it took 30.8 seconds.
>
So that is a 15% difference for full builds.  In practice, of
course, full rebuilds are rarely needed, and most builds after
changes to the source are within a second or so. 

>
Then there's something very peculiar about your codebase.
 

 
 
To me it looks more likely that your codebase is very unusual rather
than David's
 
In order to get meaningful measurements I took embedded project that
is significantly bigger than average by my standards. Here are
times of full parallel rebuild (make -j5) on relatively old
computer (4-core Xeon E3-1271 v3).
 
Option time(s) -g time text size
-O0    13.1      13.3   631648
-Os    13.6      14.1   424016
-O1    13.5      13.7   455728
-O2    14.0      14.1   450056
-O3    14.0      14.6   525380
 
The difference in time between different -O settings in my
measurements is even smaller than reported by David Brown. That can
be attributed to older compiler (gcc 4.1.2). Another difference is
that this compiler works under cygwin, which is significantly
slower both than native Linux and than native Windows. That causes
relatively higher make overhead and longer link. 

 
I don't know why Cygwin would make much difference; the native code
is still running on the same processor.
>

I don't know specific reasons. Bird's eye perspective is that cygwin
tries to emulate Posix semantics on platform that is not Posix and
achieves that by using few low-granularity semaphores in user space,
which seriously limits parallelism. Besides, there are problems with
emulation of Posix I/O semantics that cause cygwin file I/O to be 2-3
times slower that native Windows I/O. The later applies mostly to
relatively small files, but, then again, software build mostly
accesses small files.
As a matter of fact, a parallel speed up I see on this project on this
quad-core machine is barely 2x. I expect 3x or a little more for the
same project with native Windows tools.

However, is there any way of isolating the compilation time (turning
.c files into either or .o files) from 'make' the linker? Failing
that, can you compile just one module in isolation (.c to .o) with
-O0 and -O2, or is that not possible?
>

Of course, there is a way. But it's more work and gives an answer I am
not interested to know. Here are fully agree with what David said few
posts below.

Those throughputs don't look that impressive for a parallel build on
what sounds like a high-spec machine.
>

Yes, not that impressive.
However I wouldn't call the machine high-spec. More precisely, when
originally bought almost decade ago it was high-spec (but not top-spec)
machine for FPGA development. FPGA development tools even today are not
so good at parallelization of their jobs. 10 years ago parallelization
gain above 2 cores was non-existing. Also, data access patterns of this
tools tend to have poor locality of reference. It means that big L3
cache has limited usefulness. On the other hand, low-latency main
memory is very useful. This specific machine is bought with this
constraints in mind - high [for 2014] single-thread performance,
relatively small 8MB L3 cache, small [for multi-user server] 32 GB
low-latency main memory built of unbuffered DIMMs rather than of more
typical for server registered DIMMs.
People that care about parallel software builds buy quit different sort
of servers - dual-socket, lots of cores, big last level cache, big main
memory with high throughput and high latency. Back in the second half of
2014 those of them that had bigger budgets bought Xeon E5-2697 v2; those
with smaller budgets preferred Xeon E5-2697 v2. Those with good
contacts were getting Xeon E5 v3 that was already lounched but not
available for everyone.

Your processor has a CPU-mark double that of mine, which has only two
cores, and is using one.
 
Building a 34-module project with .text size of 300KB, with either
gcc 10 or 14, using -O0, takes about 8 seconds, or 37KB/second.
>

But my project has much more than 34 modules. 164 modules compiled
during build + several dozens in libraries.

Your figures show about 50KB/second.

text KB/second is hardly a good measure, esp. considering that we are
talking about different architectures. Mine is Altera Nios2 - 32-bit
RISC processor very similar to MIPS. The code density for this
architecture is significantly lower than on ARMv7 or x386 and even
somewhat lower than x86-64 and ARM64. The exact ratio depends on the
project, but 15-20% would be typical.
Also, part of text are libraries that we not compiled during this build.
But I would think that your .text size also includes libraries.

You say you use gcc 4, but an
older gcc is more likely to be faster in compilation speed than a
newer one.
>

Yes, but not dramatically so. And your ratios are extra-dramatically
different from mines, David's and Scott's.

It does sound like something outside of gcc itself.
 
For the same project, on the same slow machine, Tiny C's throughput
is 1.3MB/second. While my non-C compiler, on other projects, is
5-10MB/second, still only looking at .text segments. That is 100
times faster than your timings, for generating code that is as good
as gcc's -O0.
 
So IT IS NOT WINDOWS ITSELF THAT IS SLOW.
 
 

If I had were "native" tools then all times will be likely shorter
by few seconds and the difference between -O0 and -O3 will be close
to 10%. 

 
So two people now saying that all the many dozens of extras passes
and extra analysis that gcc -O2/O3 has to do, compared with the basic
front-end work that every toy compiler needs to do and does it
quickly, only slows it down by 10%.
 
I really don't believe it. And you should understand that it doesn't
add up.
 

I am not lying. I am pretty sure that DavidB also tells truth.
I recommend to try to compile your C compiler with gcc. Somehow I have a
feeling that -O0 to -O2 ratio you'd see will be much closer to 1.15x
than to 4x.