Re: Threads across programming languages

On 03/05/2024 01:58, Chris M. Thomasson wrote:

On 5/2/2024 4:15 PM, Lawrence D'Oliveiro wrote:

On Thu, 2 May 2024 13:28:15 -0700, Chris M. Thomasson wrote:
>

On 5/1/2024 10:39 PM, Lawrence D'Oliveiro wrote:

>
On Wed, 1 May 2024 22:20:47 -0700, Chris M. Thomasson wrote:
>

On 5/1/2024 1:34 PM, Lawrence D'Oliveiro wrote:
>

Remember, we’re talking about maximizing I/O throughput here, so CPU
is not the bottleneck.

>
It can be if your thread synchronization scheme is sub par.

>
Another reason to avoid threads.

>
Why? Believe it or not, there are ways to create _highly_ scalable
thread synchronization schemes.

>
I’m sure there are. But none of that is relevant when the CPU isn’t the
bottleneck anyway.

 The CPU can become a bottleneck. Depends on how the programmer implements things.
 

So long as your async tasks have an await call somewhere in their main
loops, that should be sufficient to avoid most bottlenecks.

>
async tasks are using threads... No?

>
No. They are built on coroutines. Specifically, the “stackless” variety.
>
<https://gitlab.com/ldo/python_topics_notebooks/-/blob/master/Generators%20&%20Coroutines.ipynb?ref_type=heads>

 So, there is no way to take advantage of multiple threads on Python? Heck, even JavaScript has WebWorkers... ;^)

Python supports multi-threading.  It uses a global lock (the "GIL") in the Python interpreter - thus only one thread can be running Python code at a time.  However, if you are doing anything serious with Python, much of the time will be spend either blocked (waiting for network, IO, etc.) or using compiled or external code (using your favourite gui toolkit, doing maths with numpy, etc.).  The GIL is released while executing such code.
Thus if you are using Python for cpu-intensive work (and doing so sensibly), you have full multi-threading.  If you are using it for IO-intensive work, you have full multi-threading.  It's not going to be as efficient as well-written compiled code, even with JIT and pypy, but in practice it gets pretty close while being very convenient and developer friendly.
If you really need parallel running of Python code, or better separation between tasks, Python has a multi-processing module that makes it simple to control and pass data between separate Python processes, each with their own GIL.