Re: xxd -i vs DIY Was: C23 thoughts and opinions

On 6/3/2024 12:39 PM, Bonita Montero wrote:

There's a good reason for sth. like #embed: I just wrote a very quick
xxd alternative and generated a C-file with a char-array and the size
of this file is 1,2GB. MSVC quitted compilation after allocating 50GB
of memory, gcc and clang compiled for minutes. This would be better
with an #embed-Tag. So there's really good reason for that.
This is my xxd-substitude. Compared to xxd it only can dump C-files.
On my PC it's about 15 times faster than xxd because it does its own
I/O-buffering.
 
#include <iostream>
#include <fstream>
#include <charconv>
#include <span>
#include <vector>
 
using namespace std;
 
int main( int argc, char **argv )
{
    if( argc < 4 )
        return EXIT_FAILURE;
    char const
        *inFile = argv[1],
        *symbol = argv[2],
        *outFile = argv[3];
    ifstream ifs;
    ifs.exceptions( ifstream::failbit | ifstream::badbit );
    ifs.open( inFile, ifstream::binary | ifstream::ate );
    streampos size( ifs.tellg() );
    if( size > (size_t)-1 )
        return EXIT_FAILURE;
    ifs.seekg( ifstream::beg );
    union ndi { char c; ndi() {} };
    vector<ndi> rawBytes( size );
    span<char> bytes( &rawBytes.data()->c, rawBytes.size() );
    ifs.read( bytes.data(), bytes.size() );
    ofstream ofs;
    ofs.exceptions( ofstream::failbit | ofstream::badbit );
    ofs.open( outFile );
    vector<ndi> rawBuf( 0x100000 );
    span<char> buf( &rawBuf.begin()->c, rawBuf.size() );
    ofs << "unsigned char " << symbol << "[" << (size_t)size << "] = \n{\n";
    auto rd = bytes.begin();
    auto wrt = buf.begin();
    auto flush = [&]
    {
        ofs.write( buf.data(), wrt - buf.begin() );
        wrt = buf.begin();
    };
    while( rd != bytes.end() )
    {
        size_t remaining = bytes.end() - rd;
        constexpr size_t N_LINE = 1 + 12 * 6 - 1 + 1;
        size_t n = remaining > 12 ? 12 : remaining;
        auto rowEnd = rd + n;
        *wrt++ = '\t';
        do
        {
            *wrt++ = '0';
            *wrt++ = 'x';
            char *wb = to_address( wrt );
            (void)(wrt + 2);
            auto tcr = to_chars( wb, wb + 2, (unsigned char)*rd++, 16 );
            if( tcr.ptr == wb + 1 )
                wb[1] = wb[0],
                wb[0] = '0';
            wrt += 2;
            if( rd != bytes.end() )
            {
                *wrt++ = ',';
                if( rd != rowEnd )
                    *wrt++ = ' ';
            }
        } while( rd != rowEnd );
        *wrt++ = '\n';
        if( buf.end() - wrt < N_LINE )
            flush();
    }
    flush();
    ofs << "};\n" << endl;
}

It would be nice to see small samplings of the files used
for input and output. Just a couple lines of the "meat" section
would be sufficient.

For anything larger, https://pastebin.com/ could hold up to 512KB
of text, for a small sample of each. Some of the USENET servers
have limits on message size. For a binary file, you can just use
the hex editor representation of the binary file.

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000000  4D 5A 50 00 02 00 00 00 04 00 0F 00 FF FF 00 00  MZP.........ÿÿ..
00000010  B8 00 00 00 00 00 00 00 40 00 1A 00 00 00 00 00  ¸.......@.......
00000020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000030  00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00  ................
00000040  BA 10 00 0E 1F B4 09 CD 21 B8 01 4C CD 21 90 90  º....´.Í!¸.LÍ!..
00000050  54 68 69 73 20 70 72 6F 67 72 61 6D 20 6D 75 73  This program mus
00000060  74 20 62 65 20 72 75 6E 20 75 6E 64 65 72 20 57  t be run under W
00000070  69 6E 33 32 0D 0A 24 37 00 00 00 00 00 00 00 00  in32..$7........

   Paul