Re: Rationale for aligning data on even bytes in a Unix shell file?

On 5/9/2025 2:20 PM, Keith Thompson wrote:

BGB <cr88192@gmail.com> writes:

On 5/9/2025 12:52 PM, Bonita Montero wrote:

Am 07.05.2025 um 12:08 schrieb BGB:
>

If you know one side is UTF-8 and the other is UTF-16, then
conversion does not need to know or care which locale is in effect.

Unicode hasn't locales, i.e. alternative meanings for the same code-
point. Even the characters from 128 to 255 are fixed to Latin-1.
>

>
A locale is not an encoding; nor is it a codepage.
>
A locale is a set of formatting and language-specific rules to apply.
>
Which, in some past contexts, may have been associated with the usage
of specific code pages, but codepages are N/A with Unicode. Even as
such, various language specific rules may still exist.
>
For things like case-folding, you may still need to care about which
language (AKA, locale) is in effect, as some conversions may apply to
some languages but not others.
>
Some letters case-map differently depending on the language, ligatures
may be in effect (which may compose/decompose or map to other
ligatures), etc.
>
>
Or, one just throws a lot of this out and uses a simplified set of
"mostly language neutral" rules.
>
Say, case conversion maps:
   Upper: 0061..007A -> 0041..005A
   Lower: 0041..005A -> 0061..007A
   Upper: 00E0..00FE -> 00C0..00DE
   Lower: 00C0..00DE -> 00E0..00FE
   ... (Add a few more, for Greek / Cyrillic / etc)
>
And, maybe a few special cases, say (*):
   009A <-> 008A
   009C <-> 008C
   009E <-> 008E
   00FF <-> 009F
*: Assuming the "1252 mappings in Unicode Space replacing C1 controls" wonk.
>
Probably ignore most everything else, it passes through as-is.

 There are a number of existing filesystems that are case-insensitive
(and mostly case-preserving): FAT, NFTS, ext4 with certain options,
etc.  Presumably all of these already have established rules for case
mapping, determining whether two given characters like 'a' and 'A'
are to be treated as the "same".  (I don't happen to know what those
rules are or how they differ from one filesystem type to another.)
 Why are you trying to invent yet another set of rules?
 https://xkcd.com/927/
 

This is not meant as a standard, more, "if I were to do it within the context of my project, how would I do it?..."
Assuming in this case, that I am doing it within the context of my own makeshift OS and a filesystem specific to said OS (for my own CPU architecture, etc). Actually, it was mainly a CPU/ISA project, OS exists mostly because no other OS exists for this arch, and generally porting one has consistently looked like the "more effort" option.
In which case, what exactly Linux or Windows do, doesn't particularly matter as much (except maybe where if they know of a particularly useful set of rules that can be scavenged; but if there are, I don't know off-hand where they are documented at).
As for implementation, internally, it would probably be to lower-case everything and then store a bitmask of which symbols were upper case (so that the original filename can be regenerated), as this is compatible with hash-tables and binary search.
Though, most likely, this would be stored at 1 bit per codepoint, and then later used to regenerate the string in the traditional format (say, for sake of operations like "readdir()").
...