Sujet : Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API
De : greg.ewing (at) *nospam* canterbury.ac.nz (Greg Ewing)
Groupes : comp.lang.pythonDate : 01. Oct 2024, 23:48:24
Autres entêtes
Message-ID : <lm391bFu38hU1@mid.individual.net>
References : 1 2 3 4 5 6
User-Agent : Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:91.0) Gecko/20100101 Thunderbird/91.3.2
On 1/10/24 8:34 am, Left Right wrote:
You probably forgot that it has to be _streaming_. Suppose you parse
the first digit: can you hand this information over to an external
function to process the parsed data? -- No! because you don't know the
magnitude yet.
By that definition of "streaming", no parser can ever be streaming,
because there will be some constructs that must be read in their
entirety before a suitably-structured piece of output can be
emitted.
The context of this discussion about integers is the claim that
they *could* be parsed incrementally if they were written little
endian instead of big endian, but the same argument applies either
way.
-- Greg