Re: Baby X is bor nagain

On 13/06/2024 16:43, Michael S wrote:

On Thu, 13 Jun 2024 13:53:54 +0200
David Brown <david.brown@hesbynett.no> wrote:
 

On 12/06/2024 23:29, Michael S wrote:

On Wed, 12 Jun 2024 15:46:44 +0200
David Brown <david.brown@hesbynett.no> wrote:
  

>
I also don't imagine that string literals would be much faster for
compilation, at least for file sizes that I think make sense.

>
Just shows how little do you know about internals of typical
compiler. Which, by itself, is o.k. What is not o.k. is that with
your level of knowledge you have a nerve to argue vs bart that
obviously knows a lot more.
  

>
I know more than most C programmers about how certain C compilers
work, and what works well with them, and what is relevant for them -
though I certainly don't claim to know everything.  Obviously Bart
knows vastly more about how /his/ compiler works.  He also tends to
do testing with several small and odd C compilers, which can give
interesting results even though they are of little practical
relevance for real-world C development work.
>

 Since he do compilers himself, he has much better feeling [that you
or me] of what is hard and what is easy, what is small and what is big,
what is fast and what is slow. That applies to all compilers except
those that are very unusual. "Major" compiler are not unusual at all.
 

I know enough about compiler design and implementation to have a pretty good idea about many parts of it, though certainly not all of it.  To be clear - theoretical knowledge is not the same as practical experience. I realise that array initialisation by a sequence of numbers has overhead (this was discussed at length in a previous thread), and that a string literal will likely have less.  But the difference is not particularly significant for realistic file sizes.

Testing a 1 MB file of random data, gcc -O2 took less than a second
to compile it.

 Somewhat more than a second on less modern hardware. Enough for me to
feel that compilation is not instant.
But 1 MB is just an arbitrary number. For 20 MB everybody would feel
the difference. And for 50 MB few people would not want it to be much
faster.
 

But what would be the point of trying to embed such files in the first place?  There are much better ways of packing large files.  You can always increase sizes for things until you get problems or annoying slowdowns, but that does not mean that will happen in practical situations.
And even if you /did/ want to embed a 20 MB file, and even if that took 20 seconds, so what?  Unless you have a masochistic build setup, such as refusing to use "make" or insisting that everything goes in one C file that is re-compiled all the time, that 20 second compile is a one-off time cost on the rare occasion when you change the big binary file.
Now, I am quite happy to agree that faster is better, all other things being equal.  And convenience and simplicity is better.  Once the compilers I use support #embed, if I need to embed a file and I don't need anything more than an array initialisation, I'll use #embed.  Until then, 5 seconds writing an "xxd -i" line in a makefile and a 20 second compile (if it took that long) beats 5 minutes writing a Python script to generate string literals even if the compile is now 2 seconds.

  One megabyte is about the biggest size I would think
makes sense to embed directly in C code unless you are doing
something very niche - usually if you need that much data, you'd be
better off with separate files and standardised packaging systems
like zip files, installer setup.exe builds, or that kind of thing.
>
Using string literals, the compile time was shorter, but when you are
already below a second, it's all just irrelevant noise.
>
For much bigger files, string literals are likely to be faster for
compilation for gcc because the compiler does not track as much
information

 And that is sort of the thing that bart knows immediately. Unlike you
and me.

I can't answer for /you/, but /I/ knew this - look through the discussions on #embed if you like.  To be fair, I did not consider string literals very much there - they are a pretty pointless alternative when #embed is coming and integer sequences are fast enough for any realistic use.

>

And I
have heard (it could be wrong) that MSVC has severe limits on the
size of string literals, though it is not a compiler I ever use
myself.
 

>

For the MSVC C compiler, I see this:
>
<https://learn.microsoft.com/en-us/cpp/c-language/maximum-string-length>
>
Each individual string is up to 2048 bytes, which can be concatenated
to a maximum of 65K in total.
>
I see other links giving different values, but I expect the MS ones
to be authoritative.  It is possible that newer versions of their C
compiler have removed the limit, just as for their C++ compiler, but
it was missing from that webpage.
>
(And I noticed also someone saying that MSVC is 70x faster at using
string literals compared to lists of integers for array
initialisation.)
>

 I didn't know it, thanks.

I didn't know the details either, until you challenged me and I looked them up!

It means that string method can't be used universally.

That depends on the state of the current MSVC compiler - and perhaps other compilers.  The C standards only require support for 4095 characters in a string literal.  (They also only require support for objects up to 32767 bytes in length - and for that size, any method should be fast.)

 Still, for C (as opposed to C++), limitation of compiler can be tricked
around by declaring container as a struct. E.g. for array of length
1234567
 struct {
   char bulk[123][10000];
   char tail[4567];
} bar = {
  {
   "init0-to-99999" ,
   "init10000-to-199999" ,
   ....
  },
  "init123400-to1234566"
};
 For that I'd expecte compilation speed almost as fast as of one string.
 

I suppose so, but it is not pretty!