Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

On 2024-10-01 at 04:46:35 +1000,
Chris Angelico via Python-list <python-list@python.org> wrote:

On Tue, 1 Oct 2024 at 04:30, Dan Sommers via Python-list
<python-list@python.org> wrote:

>
But why do I need to start with the least
significant digit?

 
If you start from the most significant, you don't know anything about
the number until you finish parsing it. There's almost nothing you can
say about a number given that it starts with a particular sequence
(since you don't know how MANY digits there are). However, if you know
the LAST digits, you can make certain statements about it (trivial
examples being whether it's odd or even).

But that wasn't the question.  Sure, under certain circumstances and for
specific use cases and/or requirements, there might be arguments to read
potential numbers as strings and possibly not have to parse them
completely before accepting or rejecting them.

And if I start with the least significant digit and the number happens
to be written in scientific notation and/or has a decimal point, then I
can't tell whether it's odd or even until I further process the whole
thing anyway.

It's not very, well, significant. But there's something to it. And it
extends nicely to p-adic numbers, which can have an infinite number of
nonzero digits to the left of the decimal:
 
https://en.wikipedia.org/wiki/P-adic_number

In Common Lisp, integers can be written in any integer base from two to
thirty six, inclusive.  So knowing the last digit doesn't tell you
whether an integer is even or odd until you know the base anyway.

Curiously, we agree:  if you move the goal posts arbitrarily, then
some algorithms that parse JSON numbers will fail.