Re: technology discussion → does the world need a "new" C ?

On 7/9/2024 10:54 AM, Michael S wrote:

On Tue, 9 Jul 2024 16:37:31 +0200
David Brown <david.brown@hesbynett.no> wrote:
 

On 06/07/2024 21:33, BGB wrote:

>
In my compiler (BGBCC), such an internal pointer exists for arrays
and structures in the local stack frame.
>
No separate pointer exists inside of things like structs, where, as
can be noted, the array exists at a fixed size and location.
>
>
So, eg:
    void Foo()
    {
       int a[100];
       ...
    }
>
There is both the space for 100 integers reserved in the stack
frame, and a variable 'a' which exists as an implicit pointer to
that location.
>
>
But, say:
    void Foo()
    {
       int a[8192];
       ...
    }
>
There is no space reserved on the stack, and the array is instead
allocated dynamically (from the heap). In this case, the "a"
variable exists as a pointer to that location in memory.
>
Similar treatment also applies to structs.
  

>
>
The C standard does not require a stack or say how local data is
implemented, it just gives rules for the scope and lifetime of
locals. However, I would be surprised and shocked to find a compiler
I was using allocate local data on the heap in some manner.  If I
have an array as local data, it is with the expectation that it is
allocated and freed trivially (an add or subtract to the stack
pointer, typically combined with any other stack frame).  If I want
something on the heap, I will use malloc and put it on the heap.
>
Such an implementation as yours is not, I think, against the C
standards
- but IMHO it is very much against C philosophy.
>

 I wouldn't mind if my ABI/compiler allocates all big local objects
together with all local VLA either on separate secondary stack or even
on heap. Such strategy will improve locality of reference for primary
stack. So, despite higher management overhead, it could sometimes be
advantageous even for performance.
The main advantage however is not performance, but reducing the
severity of damage caused by buffer overrun bugs.
 Although if "security" is a primary concern then one would want
stricter policy than the one outlined above. I.e. not only big objects,
but small object as well should be allocated away from primary stack as
long as their address participate in pointer arithmetic that can't be
proven safe by static analysis.
 

It is mostly for big objects to not lead to stack overflows...
On a 128K stack, theoretically the maximum number of 16K objects on the stack is 8 (assuming no space for anything else).
You don't want to require a big stack if:
It is possible for the system to run without an MMU, thus bigger stacks waste memory;
Most of the programs do not benefit from having a larger stack.
I did tweak the rules slightly a little more recently:
   If an object is 16K or larger, it goes on the heap;
   If the sum of objects is larger than 16K:
     Calculate n = total / 12K
     New limit is 16K/n.
So, say:
   int arr1[3072];
   int arr2[3072];
Previously, both could go on the stack (using 24K), but now they will trigger the limit to be reduced to 8K, putting both on the heap.
Though, as noted, these objects go into a linked list and will be automatically freed when the function returns.
I recently did modify the mechanism (moving the responsibility from the frontend to the backend frame-layout handling), which should hopefully reduce some of the bugs I was seeing with it.
Partly, what triggered some of this originally, was GLQuake had a buffer like:
   unsigned resampbuf[256*256];
Which, would have worked with a 1MB stack, but with a 128K stack led to an immediate overflow.
This mechanism still allows such code to exist...
For context, the array was used when uploading textures to OpenGL, as Quake used many non-power-of-2 textures, but OpenGL only allows power-of-2 texture sizes; with an accepted maximum of 256*256 as this was the limit on early graphics cards; though, newer ones can typically handle 4096*4096 or similar; currently TKRA-GL assumes a limit of 1024*1024.
The general assumption though, is that these sort of large stack objects are the exception rather than the rule, so a little extra overhead is tolerable (but, implicitly, things like interrupt handlers, etc, are not allowed to use large stack arrays).
Well, more so that in my case the interrupt handler currently has a 64K stack. Originally, it was ~ 6K (in a special SRAM area), but it was difficult to avoid overflows (with things like virtual memory code), so I ended up moving the interrupt stack to RAM.
Based on some looking, apparently early versions of Mac OS/X may have done something similar, albeit:
   With an object-size limit of 32K;
   With a stack size of 512K.
This isn't too far off in terms of proportions.
Though, it looks like more modern versions went to using bigger stacks (no information if they still do the array folding).
Can note that:
   Windows defaults to a 1MB stack size;
   Linux defaults to an 8MB stack size.
   Appears modern OSX is 4MB.
Granted, none of these systems is NOMMU.