Re: technology discussion → does the world need a "new" C ?

On 7/5/2024 8:31 AM, bart wrote:

On 05/07/2024 08:30, BGB wrote:

On 7/4/2024 8:05 PM, Lawrence D'Oliveiro wrote:

It’s called “Rust”.

>
>
If anything, I suspect may make sense to go a different direction:
   Not to a bigger language, but to a more narrowly defined language.
>
Basically, to try to distill what C does well, keeping its core essence intact.

 That would just be rearranging the deck chairs I think.
 Whatever the failings of C, it is so entrenched everywhere that it is almost impossible to dislodge. Especially if you take out all the the features that people apparently find indispensible.
 There are anyway plenty of big-name contenders which change a LOT more, and are full of modern, higher-level features that people seem to want.
 

For what I want, at this point, is not a whole lot of high-level features.

C also is the only language that is supposed to work on any kind of processor; most of the others require a machine which has 8/16/32/64-bit types and have little interest in unusual targets.
 

Yeah.
In this case, I would assume limiting it to 32 and 64 bit machines.
My hobby ISA on FPGAs is 64-bit, albeit can use 32 or 64 bit pointers.
I mostly settled on using 64-bit pointers with 48-bit address space and 16-bits for tag metadata (implicitly, the high 16 bits is ignored by normal Load/Store operations).
Though, the use of pointer-tagging is not entirely invisible to C, though, unless bounds-checking is enabled, the high 16 bits are typically zeroed for C data pointers (they are used for function pointers and link-register values though, to encode the intended operating mode for the CPU and similar).

(Actually, the nearest language to C that I have seen, is mine. It stays at about that level, but it has modern conveniences such as a module scheme and proper name-spaces. However, it is a private language, and also 1-based, case-insensitive and with non-brace syntax! It is not a contender.)
 

I would assume something that wouldn't be a massive pain for copy/paste translation.

 

*1: While not exactly that rare, and can be useful, it is debatable if they add enough to really justify their complexity and relative semantic fragility.

 C only has 1D arrays, but array elements can themselves be array types.
 The only complication, a big one, is that modern C allows a variable length for those array elements (so not known at a compile-time). This is tied in with VLAs and 'variable modified types' or whatever the feature is called.
 This applies when the multi-array data is a single block of memory.
 

At least one major C compiler has rejected VLAs entirely last I heard.
I added them experimentally, but it doesn't fully work.
I also made my compiler fall back to using them for arrays which are too big (over 16K, for sake of a 128K default stack size).
In this case, they are turned into heap allocations with automatic freeing on function return, but seemingly it is buggy and is currently prone to data corruption and memory leaks.
My attempt to port Quake3 ran into issues here as it regularly tried to use arrays over the size limit (though, some comments in the code implied that OSX did something similar, albeit with a 32K size limit; some functions though tried to use 64K or 128K local arrays, which isn't really going to fly short of boosting the stack size to 512K or 1MB).
Choice of stack size being:
   32K: Generally only small programs will fit;
   64K: Many programs fit, but some don't.
     Say, Doom can fit onto a 64K stack, Quake not so much.
   128K: Stuff generally fits (*1).
*1: Though, GLQuake and Quake3 required moving a lot of former stack arrays onto the heap.
Using larger stacks is undesirable though if one wants to support NOMMU operation, since if a program allocates 1MB and only uses 90K or so, then the remaining 934K are entirely wasted.
Generally, recursion and register save/restore by itself isn't enough to threaten a 128K stack, but large structs and arrays will do so readily.
Luckily, an ABI based around pass-by-reference has little issue with internally turning large structs into heap allocations.
Though, having a compiler which turns large arrays or large structs into a compiler error (vs giving a warning and turning them into heap allocs), probably wouldn't fly to well.
Though, in the attempted Quake3 port, did end up turning some large global structs into heap allocations as well, mostly as they caused the ".bss" section to become so large that the PE loading was failing (at the moment stuff appears to break if the total size of the PE image goes over 8MB).
Then again, there is a limit at present (in my compiler) that the handling of relocs (in the PE export code) can't express sections larger than 8MB (the remaining bits of a 32-bit value are used to encode the reloc type and section number). It is possible I was hitting this limit (it was a TODO feature to expand this part of the code to use a 64-bit format for intermediate reloc handling).

If using pointers, one almost invariably needs to fall back to doing "arr[y*N+x]"

 I've never had to do that, even if N was a variable. If using Iliffe vectors then C allows you do regular indexing even with runtime bounds.
 

Note that multidimensional indexing via multiple levels of pointer indirection would not be effected by this.

 These are Iliffe vectors.
 

Pros/cons.
For many uses, one still wants a contiguous buffer.
For some uses (typically limited to square power-of-2 arrays), I have used Morton order, but I also ended up adding special CPU instructions to deal with Morton order arrays.

 

Similarly, structs may not be declared at the point of use, but only as types.

 This is how it works on my language; they must be a named user-type. It's crazy how C allows them just anywhere, even in useless declarations like this:
    struct {int x,y;};
 

Yeah.
As-is the parser needs to lift them out and put them in their own special part of the AST.
Ideally, one doesn't want this.
Ideally also one doesn't want to need to parse a crapton of prototypes and structs/typedefs/etc for every translation unit either.

Though, would want to do a few things differently from my current IR to be able to reduce memory footprint; as my current IR was designed in such a way that it is effectively necessary to load in the whole program before code-generation can be done. Ideally one would want the compiler to be able to read-in the IR as-needed and discard the parts it is already done with (but, existing efforts to redesign the IR stage here have tended to fizzle; would effectively need to redesign the compiler backend to be able to make effective use of it).
>
For a new compiler, could make sense to try to "do it right this time" (though, not too much of an issue if running the compiler on a modern PC, as they are "not exactly RAM constrained" in this area; so reading in and decoding the IR and symbol tables for an entire executable image at the same time, is not too much of an issue).

 It's not an issue at all. My main compiler is a whole-program one; the entire source code for my biggest program occupies 1/8000th of the memory of my PC. That would be like building an 8-byte source file on my first computer.
 Presumably you are running on some restricted hardware of your own?
 

Ideally, I would want a compiler that I could run on my ISA, without needing to provide like 512MB or 1GB of swap space to do so...
One of the main FPGA boards I am targeting has 128MB of RAM, and can generally manage a CPU core running at 50MHz.

If pulled of well, such a module system could be both faster and require less memory use in the compiler if compared with headers

 Headers are a bottleneck in C. 50 modules all including the same huge header (say the 1000 #includes involved with #include <gtk<>, which is for GTK2), would involve repeating all that work 50 times.
 

Hence, unity builds...
The performance gains of compiling, say, 50-100 KLOC all in one translation unit without needing to duplicate a bunch of header parsing, more than makes up for any loss due to inability to run compiler instances in parallel (and if one does have multiple compiler instances, a decent sized program can still be reduced down to a matter of seconds of build time).

Even with unity builds, build times can still get annoying for bigger programs. And here, I am talking like 20 seconds to rebuild a 250kLOC program.

 In my language, a 250Kloc app would take about half a second to build into an executable.
 My C compiler is slower since it uses a discrete ASM stage.
 (It will build sql.c which is actually 250Kloc including headers, in half a second, generating a 1MB EXE, however that program is 40% comments, and big chunks are conditional blocks. Preprocessing it generates 85Kloc.)
 

Granted, even if parsing is fast, this still leaves the challenge of fast/efficient machine-code generation.

 Unoptimised code in my case (for the way I write code, for my language, and for my apps) is about 50% slower than when passed through C and gcc-O3.
 

On my ISA, it is slower.
Though, ironically, despite my compiler's general level of suck in these areas (vs GCC), in terms of performance it is beating out "GCC -O3" targeting RISC-V.
It was pretty close with static-linked non-relocatable images in RV64G.
Though, with PIE images, it has turned into "no contest" (PIC/PIE seems to kinda ruin both code density and performance for RISC-V).
Where, I have a CPU design that can run both my own ISA and RV64G (technically, at the same time; such as a kernel in my own ISA but programs compiled as RV64G)
This is despite also my compiler using an ABI designed around the assumption of NOMMU operation (similar to FDPIC in this sense). Where, essentially, function calling generally involves a magic ritual to reload the Global Pointer to make sure it points to the correct location for whichever image is currently executing code.
Though, I suspect much of this is because RISC-V takes a fairly steep hit to performance due to a few issues:
Lack of register-indexed memory addressing;
No good way to deal with constant/immediate values larger than 12 bits.
Say, a 32-bit constant typically requires a 2-instruction sequence to load it into a register (though, GCC seems to prefer turning all of these constants into memory loads rather than use LUI+ADD or similar); and 64 bit always gets turned into a memory load.
There are a lot of other ISA differences, but I suspect these are likely the big ones.
...
Technically, could have tried porting Quake3 to build for RV64G instead of my own ISA, but then I would need to go implement loader support for ELF Shared-Objects in my "OS", which I don't want to deal with at the moment.
Though, I am using a software OpenGL rasterizer, and past attempts to build the OpenGL rasterizer in RISC-V mode resulted in performance that could best be described as "glacial" on a 50MHz CPU (32 GPRs, no SIMD, and no RGB helper instructions, really hurt at this part).
Though, this testing was generally with GLQuake (but, wouldn't expect Quake3Arena to be much different here). Though, I still kinda have doubts that Quake3 is going to give particularly playable performance on a 50MHz CPU with software rasterized OpenGL (if/when I get it fully working).
...