Re: Is Parallel Programming Hard, And, If So, What Can You Do About It?

"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:

Only if the cores and/or "hardware threads" do not interfere with one
another? Fwiw, an example of an embarrassingly parallel algorithm is
computing the Mandelbrot set. Actually, this reminds me of the "alias"
problem with Intel hyper threading in the past.

shortly after graduating and joining IBM, I got roped into helping with
hyperthreading the 370/195. It had pipelined, out-of-order execution,
but conditional branches drained the pipeline and most code only ran
system had half rated throughput. Two hardware i-streams ... each
running at half throughput would (might) keep system full throughput.

hardware hyperthreading mentioned in this about Amdahl winning the
battle to make ACS, 360 compatible (folklore it was shutdown because IBM
was concerned that it would advance the state-of-the-art too fast and
IBM would loose control of the market, and Amdahl leaves IBM).
https://people.computing.clemson.edu/~mark/acs_end.html

Then decision was made to add virtual memory to all 370s, and it was
decided it would be too difficult to add it to 370/195 and all new 195
activity was shutdown (note operating system for 195 was MVT and its
shared memory multiprocessor support on 360/65MP was only getting
1.2-1.5 throughput of single processor, so running 195 with simulated
multiprocessor with two i-streams ... would only be more like .6 times
fully rated throughput (all hardware might be running at 100%, but the
SMP overhead would limit productive throughput); trivia the
multiprocessor overhead continues up throught MVS.

also after joining IBM, one of my hobbies was enhanced production
operating systems for internal datacenters and the online
sales&marketing support HONE systems were early (& long time)
customer. Then with decision to add virtual memory to all 370s, there
was also decision to do VM370 and in the morph of CP67->VM370 a lot of
things were simplified and/or dropped (including multiprocessor
support). I then start adding stuff back into VM370 and initially do
multiprocessor support for the HONE 168s so they can add 2nd processor
to all their systems (and managed to get twice single processor
throughput with some cache affinity hacks and other stuff).

In the mid-70s, after Future Systems implodes,
http://www.jfsowa.com/computer/memo125.htm
https://en.wikipedia.org/wiki/IBM_Future_Systems_project
https://people.computing.clemson.edu/~mark/fs.html

I get roped into helping with a 370 16-CPU multiprocessor design. It was
going fine until somebody tells head of POK (high end 370 processors)
that it could be decades before POK's favorite son operating system (now
"MVS") had ("effective") 16-cpu support (POK doesn't ship a 16-CPU
system until after the turn of the century) ... and some of us are
invited to never visit POK again.

--
virtualization experience starting Jan1968, online at home since Mar1970