Re: technology discussion → does the world need a "new" C ?

On 7/18/2024 2:46 AM, David Brown wrote:

On 17/07/2024 19:53, BGB wrote:

On 7/17/2024 6:38 AM, Bart wrote:

On 13/07/2024 10:39, BGB wrote:
>

But, as I see it, no real point in arguing this stuff (personally, I have better stuff to be doing...).

>
We all do. But this group seems to be about arguing about pointless stuff and you might come here when you want a respite from proper work.
>
However (here I assume you've gone back to Quake but that other interested parties might be reading this), consider the program below.
>

>
I got back to debugging...

 To be clear - you are talking about debugging your compiler here, yes?
 

My compiler and my Quake 3 port, but most of the bugs in the Quake 3 port thus far were due to bugs either in my compiler or in the runtime libraries.

>
Ironically, one of the big bugs I ended up finding was related to internal struct handling "leaking through" and negatively effecting stuff.
>
say:
   typedef struct foo_s foo_t;  // don't care what it contains for now.
>
   foo_t arr[...];
   foo_t temp;
   int i, j;
   ...
   temp=arr[i];
   arr[i]=arr[j];
   arr[j]=temp;
>
Internally, it would load a reference to arr[i] into temp, but then this location would get overwritten before the third assignment happened, causing the incorrect contents to be copied to arr[j].
>
For now, have ended up changing stuff such that any struct-assignment (for structs in the "by-ref" category) to a local variable will instead copy the contents to the memory location associated with that struct.

 How could it possibly mean anything else?  Structs in C are objects - contiguous blocks of bytes interpreted by a type.  Assigning them will mean copying those bytes.  Pretending the language sometimes means structs and sometimes means magical auto-dereferencing pointers to structs is simply wrong.
 

The "magical auto dereferencing pointers" interpretation gives better performance, when it works. In this case, it didn't work...
Sadly, there is no good way at the moment to know whether or not it will work, for now forcing the slower and more conservative option.
Because, it turned out this was buggy and Quake 3 essentially stepped on this issue here.
The bug wasn't the intended behavior in this case, but was in effect a bug.

If "foo_t" is 2000 bytes long, then "foo_t temp" makes a 2000 byte space in your local variables (the stack, on virtually every platform) and "temp = arr[i];" does a 2000 byte memcpy().  The same thing applies if "foo_t" is 2 bytes long, or 2 megabytes long.  And if there is a stack overflow making "temp", that's the programmer's problem.
 

For now:
1 - 16 bytes, goes in registers, except when accessing a member where it needs to be in-memory; unless it is a SIMD type which is special and allows accessing members with the value still in registers.
17 bytes to 15.999K: Accessed by an implicit reference, uses hidden copying to mimic by-value semantics (not quite foolproof as of yet it seems).
16K and beyond, quietly turned into a heap allocation (with a compiler warning). Should otherwise look the same as the prior case.

It is only once that is all working properly and reliably that you can begin to think about optimisation - code that gives the same observable behaviour "as if" you had direct translation, but is more efficient. Maybe your target architecture has a way to make a "memswap()" function that is more efficient than two "memcpy()" calls, and you can use that.
 

I did implement such a function, and ended up using it in the "qsort()" function (also ended up beating on this as it was also misbehaving).
<----
/*
Swap two items in memory.
  */
void _memswap(void *ptra, void *ptrb, int sz)
{
u64 v0, v1, v2, v3;
byte *csa, *csb;
int n;

csa=ptra;
csb=ptrb;
n=sz;

while(n>=16)
{
v0=((u64 *)csa)[0];
v2=((u64 *)csb)[0];
v1=((u64 *)csa)[1];
v3=((u64 *)csb)[1];
((u64 *)csb)[0]=v0;
((u64 *)csa)[0]=v2;
((u64 *)csb)[1]=v1;
((u64 *)csa)[1]=v3;
csa+=16; csb+=16;
n-=16;
}
if(n>=8)
{
v0=((u64 *)csa)[0];
v2=((u64 *)csb)[0];
((u64 *)csb)[0]=v0;
((u64 *)csa)[0]=v2;
csa+=8; csb+=8;
n-=8;
}
if(!n)
return;
if(n>=4)
{
v0=((u32 *)csa)[0];
v2=((u32 *)csb)[0];
((u32 *)csb)[0]=v0;
((u32 *)csa)[0]=v2;
csa+=4; csb+=4;
n-=4;
}

if(n)
{
while(n>0)
{
v0=csa[0]; v1=csb[0];
csb[0]=v0; csa[0]=v1;
csa++; csb++;
n--;
}
}
}
---->
But, Quake 3 had used struct-assignment for swapping values, and had stepped on a bug.
Previously ported code did not step on the bug because none of it did this.

And it is only when the direct translation is working properly that you can start to think of improving user convenience.  Perhaps you could allocate large temporary objects in non-stack memory somewhere to avoid stack overflows.  I don't think that is a good idea, but it's a possibility.  Giving compiler warnings about large stack objects is a much better solution IMHO.
 

It both warns and also turns it into a heap allocation.
Because warning and the code still working, is better than warning and the program most likely crashing due to a stack overflow (and in cases with no memory protection, probably overwriting a bunch of other stuff in the process).
Also for "arcane technical reasons", as of yet I can't put program stacks in pagefile backed memory when using the MMU.
But, as noted, compiler was not written with any sort of formal design process. And, it was not a top-down (nor planned in advance) design, but more emerged incrementally (roughly over the past 15 years).
Technically, I really suck at any sort of "top down" thinking, so don't really use it; pretty much everything is more of a bottom-up / incremental process (including my thought processes and approach to life in general, always more "what do I have on hand at the moment?").
But, development was more like:
Originally (long ago), it was essentially a C reskin parser on top of a JavaScript-like core (IOW: internally using dynamic types for anything "non-trivial"), but it sucked so was shelved for a number of years (the compiler was then repurposed mostly as a "parse headers and generate FFI glue" tool).
At the time, the new fangled technology I was working with was mostly bytecode and some amount of dynamic native-code generation (initially partly inspired by what was going on in Quake 3's QVM).
Initial results weren't very good (barely worked at all for compiling C code, mostly limited to a simplistic coding style).
It was partly a fork off of a version of my interpreter which had previously used a direct AST walking interpreter (slow), but by that point I was mostly moving away from AST walking and over to bytecode (so the main part that was carried over from the VM was mostly the tokenizer and parser and similar...).
Then it was revived (some years later), with the intent of targeting a Dalvik-like VM, using a modified C like language (using C# style declaration parsing rather than actual C style).
Then, I wrote a backend to target SH-4, using parts of the prior VM backend. Switched over to actual C, got enough working to get Quake and then Doom working. For targeting a bare CPU, and porting these programs, kinda needed "actual C".
The SH-4 ISA mutated into BJX1, which was then partly rebooted into BJX2, which eventually mutated further eventually into its current form.
Porting Dhrystone required adding things like K&R style declaration parsing, ...
Then ported Hexen, followed by ROTT and Heretic.
Then recently ported Quake 3. Quake 3 stepped on some new issues which were missed in the prior programs.
New programs may step on new edge cases that were either missing or hacked over; often pointing out where bugs or deficiencies remain.
It was not the goal to design things around bugs, this just sorta happens sometimes.
Often, things are more "what sort of behavior does it appear the program expected?..." that influences more subtle design choices