Re: How to convert <binaryGlowMixedWithASCII> to pure ASCII

On 12/05/2025 at 15:03, Kenny McCormack wrote:

In article <vvssf0$13ls6$1@dont-email.me>,
Nuno Silva  <nunojsilva@invalid.invalid> wrote:
...

My guess is that this isn't an apostrophe, but a "right single quotation
mark", which is sadly a common sight in such a context, and Emacs tells
me that this (UCS codepoint 0x2019) is represented as E2 80 99 in UTF-8.

 Correct, but as far as I am concerned, they are all single quotes, just
mangled versions of same.  The goal is to convert them all back into
regular single quotes.  And, as you will see below, similar comments apply
for double quotes.
 The AWK code that I am currently using to clean this problem contains these
lines:
      gsub(/=..=..=9[CD]/,"\"")
     gsub(/=..=..=../,"'")
 which is good enough for me.

I use sed:
# sq = single quote, smart left single quote, smart right single quote, double quote, smart left double quote, smart right double quote
# [$sq] to change smart quotes to single quotes
$ sq=$(echo -ne '\u0027\u2018\u2019\u0022\u201c\u201d')
$ sed -e "s/[$sq]/'/g" filename

 

Are there good ways to convert such chars to something more reasonable?
The only thing that occurs to me right now is passing it through iconv
to a more limited charset using transliteration (e.g. "iconv -f utf8 -t
iso8859-1//TRANSLIT -c") and then back to the desired encoding and
charset.

 As mentioned in the OP, I have never been successful in getting "iconv" to do
much of anything.  No, this is not a plea for help or for man pages to be
read out loud.
 

(But I suppose if this is already involving perl, then perhaps such a
modification can be done through perl too.)

 Probably, but I'm not much into Perl.  I do appreciate the solution given
here by Chuck, but don't intend on doing any real deconstruction on it.
 

--
Chris Elvidge, England
I WILL NOT SPANK OTHERS