Re: ASCII to ASCII compression.

On Mon, 10 Jun 2024 07:12:57 +0100
Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:

On 10/06/2024 01:45, Lew Pitcher wrote:

On Thu, 06 Jun 2024 17:25:37 +0100, Malcolm McLean wrote:
 

Not strictly a C programming question, but smart people will see
the relavance to the topicality, which is portability.
>
Is there a compresiion algorthim which converts human language
ASCII text to compressed ASCII, preferably only "isgraph"
characters?
>
So "Mary had a little lamb, its fleece was white as snow".
>
Would become
>
QWE£$543GtT£$"||x|VVBB? 

 
I'm afraid that you have conflicting requirements here. In effect,
you want to take an array of values (each within the range of
0 to 127) and
a) make the array shorter ("compress it"), and
b) express the individual elements of this shorter array with
    a range of 96 values ("isgraph() characters")
 
Because you reduce the number of values each result element
can carry, each result element can only express a fraction
(96/128'ths) of the corresponding source element. Thus,
with the isgraph() requirement, the result will take /more/
elements to express the same data as the source did.
 
However, you want /compression/, which implies that you want
the result to be smaller than the source. And, therein lies
the conflict.
 
Can you help clarify this for me?

 > 
We have a fixed Huffman tree which is part of the algorithm and
optmised for ASCII. And we take each line otext, and comress it to a
binary string, using the Huffman table. The we code the binary string
six bytes ar a time using a 64 character dubset of ASCCI. And the we
append a special character which is chosen to be visually
distinctive..
 
So the inout is
 
Mary had a little lamb,
it's fleece was white as snow,
and eveywhere that Mary went,
the lamb was sure to. go.
 
And we get the output.
 
CVbGNh£-H$£*MMH&-VVdsE3w2as3-vv$G^&ggf-
 
 
And if it shorter or not depends on whether the fixed Huffman table
is any good.
 

Take something that is a little bigger than a text above. It does not
have to be much bigger. One page from any book  will do ("Alice's
Adventures in Wonderland" is used most often for that purpose).
Apply your compression procedure.
Then run automatic test that applies all possible single bit flips,
de-compresses and count # of mismatches vs original text. The test will
report the case with maximal # of mismatches.
Look at most corrupted text.
If your fixed Huffman table is any good, you'll see that output is
corrupted rather seriously, most likely at least one sentence will be
unrecognizable.
Alternatively, if your fixed Huffman table is no good, you output will
be as big or bigger than the input.

Popular corpus of samples for compression tests:
https://corpus.canterbury.ac.nz/descriptions/
http://corpus.canterbury.ac.nz/resources/cantrbry.zip