Re: Byte Addressability And Beyond

According to EricP  <ThatWouldBeTelling@thevillage.com>:

Ok, you accept international character data, you just don't have to
check >127 characters for "drop table" etc commands.
>
I don't think you are being paranoid enough.
I still think you have to validate or sanitize the >127 string to
ensure the code sequences only contain well formed characters.

If you're sending the strings to a database, the database will
invariably do detailed string validation so I wouldn't bother, but be
prepared for the error code if it rejects the string,

Random hack thought #1: if the string I send starts with an umlaut as
the first code point, ...

A bare umlaut displays just fine.  But see below.

Random hack thought #2: If a character has multiple combiner code points,
does changing the order create a different character or do they map to
the same display character? Or worse, maybe combiner code point order
sensitivity is character dependent, some are, some are not.

Unicode has normalization forms that deal with this. The most common
are NFC which uses precomposed combined characters, and NFD where
they're all separate (Composed and Decomposed.) NFD puts the combiners
in a well defined order.   Sensible people put all their strings into
NFC or NFD before doing anything else with them.

https://www.unicode.org/reports/tr15/
--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly