Re: Byte Addressability And Beyond

On 09/05/2024 03:24, John Levine wrote:

According to Lawrence D'Oliveiro  <ldo@nz.invalid>:

On Wed, 8 May 2024 02:47:46 -0000 (UTC), John Levine wrote:
>

It doesn't make sense to say that character strings are big- or little-
endian.

>
Yes it does, for just about any encoding other than UTF-8. Thus, you have
UTF16BE, and UTF16LE.

 Not really, those are byte orders within a character, not order of characters.
 

Or rather, they are byte orders used by different encodings of code points.  ("Characters" in Unicode are more complicated - nothing is ever simple in Unicode!)  There are no endian issues between code points, and a "string" as far as Unicode is concerned would be a sequence of code points.  You only have endian issues if you want to store these 21-bit integers in a format that is encoded in smaller lumps (like byte-addressed memory).

If you look at surrogates, you can UTF16 is big-endian.  First there's the high
surrogate, then the low one.
 There's a reason that every encoding other than UTF-8 is dead.  Who needs the grief?

Indeed.
UTF-32 is fine for internal use, however - using whatever endianness your processor prefers.  The trick is never to let it leave the one computer in any encoding other than UTF-8.