Re: Is Parallel Programming Hard, And, If So, What Can You Do About It?

On Mon, 19 May 2025 21:33:11 +0000, quadibloc wrote:

Parallel programming isn't "hard" or "easy".

Concurrent processes/threads, running either fully concurrently on
separate processors, or with asynchronous preemption on shared processors,
are hard to program. That’s where the term “heisenbug” came from, where
you had subtle, intermittent, timing-related misbehaviour that, for
example, tended to go away when you changed the code to try to narrow the
problem down.

In the 1990s, multithreading became popular, and people wanted to use it
for everything -- even GUIs. That turned out to be a bad idea. So nowadays
we recognize that threads are mainly useful to increase performance for
CPU-bound code, and not much else.

We also now have the async/await paradigm, also known as “stackless
coroutines”. Like threads, this lets us manage multiple concurrent
activities at once. Unlike threads, preemption is always explicit, which
gets rid of most possibilities for race conditions. This technique is very
useful where the performance bottleneck lies elsewhere than the CPU: i.e.
the filesystem, network, or just waiting for the user to click the mouse
or press the next key.