Re: xxd -i vs DIY Was: C23 thoughts and opinions

On 29/05/2024 13:23, Michael S wrote:

On Wed, 29 May 2024 12:23:51 +0100
bart <bc@freeuk.com> wrote:

So, let's rewrite our tiny app with fread().

>

real    0m0.577s
user    0m0.000s
sys     0m0.000s
>
152.8 MB/s. That's much better. Some people would even say that it
is good enough.

>
I now get:
>
mcc      2.3 seconds
gcc      1.6
tcc      2.3
lccwin32 2.9
DMC      2.9
>

 Mine was with MSVC from VS2019. gcc on msys2 (ucrt64 variant) should be
identical.
I wonder why your results are so much slower than mine.
Slow write speed of SSD or slow CPU?

You'd need to isolate i/o from the data processing to determine that.
However, the fastest timing on my machine is 1.4 seconds to read 100MB and write 360MB.
Your timing is 0.6 seconds to read 88MB and write, what, 300MB of text?
The difference is about 2:1, which is not that unusual given two different processors, two kinds of storage device, two kinds of OS (?) and two different compilers.
But remember that a day or two ago, your original program took over 4 seconds, and it now takes 1.6 seconds (some timings are 1.4 seconds, but I think that's the C port of my code).
(BTW I guess that superimposing your own faster buffer is not consdered cheating any more!)

You might remember that the last revised version of your test,
compiled with gcc, took 3.6 seconds, of which 2 seconds was reading
the file a byte at a time took 2 seconds.
>
By using a 128KB buffer, you get most of the benefits of reading the
whole file at once

 I hope so.
 

(it just lacks the simplicity).

 The simplicity in your case is due to complexity of figuring out the
size of the file and of memory allocation and of handling potential
failure of memory allocation all hidden within run-time library of you > language.

Yes, that moves those details out of the way to keep the main body of the code clean.
Your C code looks chaotic (sorry), and I had quite a few problems in understanding and trying to modify or refactor parts of it.
Below is the main body of my C code. Below that is the main body of your latest program, not including the special handling for the last line, that mine doesn't need.
------------------------------------------
   while (n) {
       m = n;
       if (m > perline) m = perline;
       n -= m;
       p = str;
       for (int i = 0; i < m; ++i) {
           bb = *data++;
           s = numtable[bb];
           slen = numlengths[bb];
           *p++ = *s;
           if (slen > 1)
               *p++ = *(s+1);
               if (slen > 2)
                   *p++ = *(s+2);
           *p++ = ',';
       }
       *p++ = '\n';
       fwrite(str, 1, p-str, f);
   }
------------------------------------------
   for (;;) {
     enum { BUF_SZ = 128*1024 };
     unsigned char inpbuf[BUF_SZ];
     size_t len = fread(inpbuf, 1, BUF_SZ, fpin);
     for (int i = 0; i < (int)len; ++i) {
       unsigned char* dec = bin2dec[inpbuf[i] & 255];
       memcpy(outptr, dec, MAX_CHAR_PER_NUM);
       outptr += dec[MAX_CHAR_PER_NUM];
       if (outptr > &outbuf[ALMOST_FULL_THR]) { // spill output buffer
         *outptr++ = '\n';
         ptrdiff_t wrlen = fwrite(outbuf, 1, outptr-outbuf, fpout);
         if (wrlen != outptr-outbuf) {
           err = 2;
           break;
         }
         outptr = outbuf;
       }
     }
     if (err || len != BUF_SZ)
       break;
   }
------------------------------------------