Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

Liste des GroupesRevenir à cl python 
Sujet : Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API
De : greg.ewing (at) *nospam* canterbury.ac.nz (Greg Ewing)
Groupes : comp.lang.python
Date : 01. Oct 2024, 22:48:24
Autres entêtes
Message-ID : <lm391bFu38hU1@mid.individual.net>
References : 1 2 3 4 5 6
User-Agent : Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:91.0) Gecko/20100101 Thunderbird/91.3.2
On 1/10/24 8:34 am, Left Right wrote:
You probably forgot that it has to be _streaming_. Suppose you parse
the first digit: can you hand this information over to an external
function to process the parsed data? -- No! because you don't know the
magnitude yet.
By that definition of "streaming", no parser can ever be streaming,
because there will be some constructs that must be read in their
entirety before a suitably-structured piece of output can be
emitted.
The context of this discussion about integers is the claim that
they *could* be parsed incrementally if they were written little
endian instead of big endian, but the same argument applies either
way.
--
Greg

Date Sujet#  Auteur
30 Sep 24 * Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API10Left Right
1 Oct 24 `* Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API9Greg Ewing
2 Oct 24  +* Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API2<avi.e.gross
2 Oct 24  i`- Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API1Greg Ewing
2 Oct 24  +- Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API1Left Right
2 Oct 24  +- Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API1Chris Angelico
3 Oct 24  +- Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API1Chris Angelico
3 Oct 24  +* Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API2Left Right
3 Oct 24  i`- doRe: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API1Greg Ewing
3 Oct 24  `- Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API1Left Right

Haut de la page

Les messages affichés proviennent d'usenet.

NewsPortal