On 9/13/2024 10:30 AM, David Brown wrote:
On 12/09/2024 23:14, BGB wrote:
On 9/12/2024 9:18 AM, David Brown wrote:
On 11/09/2024 20:51, BGB wrote:
On 9/11/2024 5:38 AM, Anton Ertl wrote:
Josh Vanderhoof <x@y.z> writes:
anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>
<snip lots>
Would be nice, say, if there were semi-standard compiler macros for various things:
>
Ask, and you shall receive! (Well, sometimes you might receive.)
>
Endianess (macros exist, typically compiler specific);
And, apparently GCC and Clang can't agree on which strategy to use.
>
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
...
#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
...
#else
...
#endif
>
Works in gcc, clang and MSVC.
>
>
Technically now also in BGBCC, since I have just recently added it.
Good idea.
>
>
And C23 has the <stdbit.h> header with many convenient little "bit and byte" utilities, including endian detection:
>
#include <stdbit.h>
#if __STDC_ENDIAN_NATIVE__ == __STDC_ENDIAN_LITTLE__
...
#elif __STDC_ENDIAN_NATIVE__ == __STDC_ENDIAN_BIG__
...
#else
...
#endif
>
>
This is good at least.
>
Though, generally takes a few years before new features become usable.
Like, it is only in recent years that it has become "safe" to use most parts of C99.
>
Most of the commonly used parts of C99 have been "safe" to use for 20 years. There were a few bits that MSVC did not implement until relatively recently, but I think even have caught up now.
Until VS2013, the most one could really use was:
// comments
long long
Otherwise, it was basically C90.
'stdint.h'? Nope.
Ability to declare variables wherever? Nope.
...
After this, it was piecewise.
Though, IIRC, still no VLAs or similar.
There are only two serious, general purpose C compilers in mainstream use - gcc and clang, and both support almost all of C23 now. But it will take a while for the more niche tools, such as some embedded compilers, to catch up.
<stdbit.h> is, however, in the standard library rather than the compiler, and they can be a bit slow to catch up.
FWIW:
I had been adding parts of newer standards in my case, but it is more hit/miss (more adding parts as they seem relevant).
>
Whether or not the target/compiler allows misaligned memory access;
If set, one may use misaligned access.
>
Why would you need that? Any decent compiler will know what is allowed for the target (perhaps partly on the basis of compiler flags), and will generate the best allowed code for accesses like foo3() above.
>
>
Imagine you have compilers that are smart enough to turn "memcpy()" into a load and store, but not smart enough to optimize away the memory accesses, or fully optimize away the wrapper functions...
>
Why would I do that? If I want to have efficient object code, I use a good compiler. Under what realistic circumstances would you need to have highly efficient results but be unable to use a good optimising compiler? Compilers have been inlining code for 30 years at least (that's when I first saw it) - this is not something new and rare.
Say, you are using a target where you can't use GCC or similar.
Say:
BJX2, haven't ported GCC as it looks like a pain;
Also GCC is big and slow to recompile.
6502 and 65C816, because these are old and probably not worth the effort from GCC's POV.
Various other obscure/niche targets.
Say, SH-5, which never saw a production run (it was a 64-bit successor to SH-4), but seemingly around the time Hitachi spun-out Renesas, the SH-5 essentially got canned. And, it apparently wasn't worth it for GCC to maintain a target for which there were no actual chips (comparably the SH-2 and SH-4 lived on a lot longer due to having niche uses).
So, for best results, the best case option is to use a pointer cast and dereference.
>
For some cases, one may also need to know whether or not they can access the pointers in a misaligned way (and whether doing so would be better or worse than something like "memcpy()").
>
Again, I cannot see a /real/ situation where that would be relevant.
I can think of a few.
Most often though it is in things like data compression/decompression code, where there is often a lot of priority on "gotta go fast".
>
Whether or not memory uses a single address space;
If set, all pointer comparisons are allowed.
>
Pointer comparisons are always allowed for equality tests if they are pointers to objects of compatible types. (Function pointers cannot be compared at all.)
>
For other relational tests, the pointers must point to sub-objects of the same aggregate object. (That means they can't be null pointers, misaligned pointers, invalid pointers or pointers going nowhere.) This is independent of how the address space(s) are organised on the target machine.
>
What you /can/ do, on pretty much any implementation with a single linear address space, is convert pointers to uintptr_t and then compare them. There may be some targets for which there is no uintptr_t, or where the mapping from pointer to integer does not match with the address, but that would be very unusual.
>
I can't think when you would need to do such comparisons, however, other than to implement memmove - and library functions can use any kind of implementation-specific feature they like.
>
>
Yeah.
>
My "_memlzcpy()" functions do a lot of relative comparisons (more than needed for memmove):
dst<=src: memmove
(dst-src)>=sz: memcpy
(dst-src)>=32: can copy with 32B blocks
(dst-src)>=16: can copy with 16B blocks
(dst-src)>= 8: can copy with 8B blocks
1/2/4: Generate a full-block fill pattern
3/5/6/7: partial fill pattern (16B block with irregular step)
>
If this is something for your library for your compiler, then of course you are free to do anything you want here - standard library code does not need to be portable, but is free to use any kind of compiler "magic" it likes. (For example, gcc has lots of builtins and extensions that are not targeted at normal code, but are targeted specifically at library writers.)
It is part of my C library, but also used for LZ decompression, which is used quite extensively.
Having a library function for this allows consolidating it from other contexts where I need this particular sort of copy function.
Granted, would not work for something like LZSS (which uses an explicit modulo window rather than the sliding window typical in LZ77 variants), but I am not using LZSS.
There is a difference here between "_memlzcpy()" and "_memlzcpyf()" in that:
the former will always copy an exact number of bytes;
the latter may write 16-32 bytes over the limit.
It may do /what/ ? That is a scary function!
This is why the latter have an 'f' extension (for "fast").
There are cases where it may be desirable to have the function write past the end in the name of speed, and others where this would not be acceptable.
Hence why there are 2 functions.
The main intended use-case for _memlzcpyf() being use for match-copying in something like my LZ4 decoder, where one may pad the decode buffer by an extra 32 byes.
Also my RP2 decoder works in a similar way.
I have a checksum algo that also works in a similar way (in terms of 16 byte blocks, but can't be used directly unless the data is or is padded to a multiple of 16 bytes).
For some use-cases, did end up needing to use a different variant which combines this as a faster "bulk checksum" and then falls back to a slower byte-for-byte algorithm for the last (sz&15) bytes, combining these into a single checksum.
Both are based on a combination of "sum and sum-of-sums" approach, which seems to be "sufficiently robust" and is relatively fast to calculate.
A very simplified notion of the algorithm:
u32 *cs, *cse;
u64 sum1, sum2;
sum1=1; sum2=0;
cs=(u32 *)buf; cse=cs+(size>>2);
while(cs<cse)
{
sum1+=*cs++;
sum2+=sum1; //magic happens here
}
sum1=((u32)sum1)+(sum1>>32);
sum2=((u32)sum2)+(sum2>>32);
sum1=((u32)sum1)+(sum1>>32);
sum2=((u32)sum2)+(sum2>>32);
return(sum1^sum2);
Can be made a little faster by effectively running 4 sets of sums in parallel (then merging the sums at the end).
While it is a little slower than a direct linear sum, it adds a little more resistance against various common cases (such as permutations of different bytes that just happen to have the same net sum). Also, adding the data as 32-bit chunks into 64 bit accumulators (and folding the bits back over at the end) makes it more resistant against some kinds of arithmetic anomalies.
Generally also much faster than a CRC or Adler32 checksum...
Say, for example, potentially faster but unacceptably weak:
u64 *cs;
u64 sum;
sum=0;
cs=(u64 *)buf; cse=cs+(size>>3);
while(cs<cse)
{ sum+=*cs++; }
//done...
>
Possible:
__MINALIGN_type__ //minimum allowed alignment for type
>
_Alignof(type) has been around since C11.
>
>
_Alignof tells the native alignment, not the minimum.
It is the same thing.
Not necessarily, it wouldn't make sense for _Alignof to return 1 for all the basic integer types. But, for" minimum alignment" it may make sense to return 1 for anything that can be accessed unaligned.
>
Where, _Alignof(int32_t) will give 4, but __MINALIGN_INT32__ would give 1 if the target supports misaligned pointers.
>
The alignment of types in C is given by _Alignof. Hardware may support unaligned accesses - C does not. (By that, I mean that unaligned accesses are UB.)
The point of __MINALIGN_type__ would be:
If the compiler defines it, and it is defined as 1, then this allows the compiler to be able to tell the program that it is safe to use this type in an unaligned way.
This also applies to targets where some types are unaligned but others are not:
Say, if all integer types 64 bits or less are unaligned, but 128-bit types are not.
There is also an "__unaligned" keyword, but this is also non-standard and mostly only effects 128 bit types in my case.
>
>
Maybe also alias pointer control:
__POINTER_ALIAS__
__POINTER_ALIAS_CONSERVATIVE__
__POINTER_ALIAS_STRICT__
>
Where, pointer alias can be declared, and:
If conservative, then conservative semantics are being used.
Pointers may be freely cast without concern for pointer aliasing.
Compiler will assume that "non restrict" pointer stores may alias.
If strict, the compiler is using TBAA semantics.
Compiler may assume that aliasing is based on pointer types.
>
>
Faffing around with pointer types - breaking the "effective type" rules - has been a bad idea and risky behaviour since C was standardised. You never need to do it. (I accept, however, that on some weaker or older compilers "doing the right thing" can be noticeably less efficient than writing bad code.) Just get a half- decent compiler and use memcpy(). For any situation where you might think casting pointer types would be a good idea, your sizes are small and known at compile time, so they are easy for the compiler to optimise.
>
>
It depends.
>
In some things, like my ELF and PE/COFF program loaders, the code can get particularly nasty in these areas...
It may look simpler in the code to do this kind of thing, but it is not /necessary/ and it is not safe unless you are writing non-portable code and are sure it will only be used on a compiler that supports it. Thus the Linux kernel requires "-fno-strict-aliasing", because some of the Linux kernel authors write crap C code. (Or, to be a bit fairer, some of the code in the Linux kernel is very old and comes from a time when writing things correctly while generating efficient results would need more effort.)
Most of this is being compiled by BGBCC for a 50 MHz cPU.
So, the CPU is slow and the compiler doesn't generate particularly efficient code unless one writes it in a way it can use effectively.
Which often means trying to write C like it was assembler and manually organizing statements to try to minimize value dependencies (often caching any values in variables, and using lots of variables).
In this case, the equivalent of "-fwrapv -fno-strict-aliasing" is the default semantics.
Generally, MSVC also responds well to a similar coding style as used for BGBCC (or, as it more happened, the coding styles that gave good results in MSVC also tended to work well in BGBCC).
And as a general rule, if you feel you really want to break the rules of C and still get something useful out at the end, use "volatile" liberally.
>
>
I have used "volatile" here to good effect.
>
>
>
Computer architects leaving Intel...
Re: Computer architects leaving Intel...
