Re: Is Parallel Programming Hard, And, If So, What Can You Do About It?

On 5/25/2025 1:05 PM, Waldek Hebisch wrote:

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> wrote:

On 5/23/2025 2:03 PM, Stefan Monnier wrote:

Stephen Fuld [2025-05-23 08:28:44] wrote:

On 5/22/2025 5:18 PM, Waldek Hebisch wrote:

It is pretty clear that due to drive mechanics track cache/buffer
is useful.

Pretty clear to everyone except one person. :-)

>
🙂
>

However, the real question is about size: how big
should it be.  For "consumer" drives I see claims of 256 MB
cache.  Given rather optimistic 200 MB/s transfer rate it is
about 1.25s of drive data, that is 80-150 rotations.  I would
expect that say 4 tracks should be enough for reading.  For
writing one could use few more tracks.  Still, advertised cache
sizes seem to be much bigger than necessary.

It's not just the rotations, but the seek time. So your example is fewer
"operations" than the 80-150 you get when just including rotations.

>
I don't understand what you're getting at, here.
I think Waldek's argument is that 256MB corresponds approximately
to the amount of data stored in 80-150 tracks, and seek time doesn't
change that fact.

>
Yes, I didn't express myself well.  :-(  And once again, I have to say
that my information may be obsolete.
>
I think it is useful to separate talking about read data from write
data.  For read data, as with any cache, more is always better than
less, though with diminishing returns.  Why pick 1.25 sec as the "cut
off point"?  If the host re-references data that it hasn't read for say
3 seconds, having it in cache still saves, probably a seek time and on
average 1/2 rotation time.  Plus, it means the heads will be free to
handle other requests.  All of this is standard cache benefits.  I see
no reason to limit the cache size and reduce this benefit.

 We are talking here about common case, that is when disc is accessed
via OS cache.  OS cache is significantly larger than disc cache, so
hit ratio for data sent to host is going to be quite low.  Disc
cache has an advantage: it gets "for free" some data that host did
not request.  But it is rather unlikely that keeping such data
for long time has significant advantage.
 

And if you are caching writes, more cache gives you more blocks to choose
from when optimizing the write back order, which reduces the time to write
them all back.

>
IIUC, for SATA drives, NCQ is still limited to 32 in-flight commands, so
unless the drive is allowed to do write-back caching it seems the amount
of space used for write-buffering is likely small (compared to 256MB).
[ Unless it is common for individual write commands to cover multi-MB
    chunks of data?  ]

>
>
For write data, I was unaware of the 32 operation limit.  I was used to
SCSI, which, IIRC was larger, and for server type applications, where
some sort of UPS is more common, the site may choose to enable write
caching in the disk.  For a disk vendor, given the small cost of the
DRAM, it is an easy choice.

 I do not look at details of disc protocol.  But with protocal done
right host would first transfer commands and then deliver data
in order requested by the drive.  So most buffering would be in
the host and disc would need just enough buffering to ensure
smooth transmission and low interrupt rate.  4 track looks like
plenty for this purpose.
 

The larger DRAM is a small component of drive cost, so the
manufacturers think it is worth including more.

>
In some markets (e.g. home routers), the size of DRAM seems to be enough
of a cost factor that it took many years until reaching 256MBs, even
though those boxes *need* that RAM for all kinds of purposes (the 128MB
of my current home-router seems to be its main source of instability).
but HDDs are pretty damn expensive beasts nowadays (because prices have
not gone down for the last 10 years or so), so I guess that makes
the relative cost of 512MB of DRAM "negligible"?

>
I can't comment on routers, but for disks, while the cost of the disk
may not have come down, increasing capacity allows reduced cost per
gigabyte.  A substantial portion of the cost is not subject to Moore's
law (e.g. drive motor, magnets and arm assembly, etc.) and some capacity
increasing technologies cost more (but not enough more to overwhelm the
capacity advantage).

 In nineties I read that for motherboard manufactures 1 cent was
"negligible", but 10 cents was significant: In volume transactions
margins were low and no party were willing to absorb 10 cents
per piece "loss".  Discs probably are less competitive than
motherboards, but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
 

Dunno, I would maybe expect an 8 or 16MB chip, unless either:
Tracks have become so large that large RAM is needed to deal with them;
These larger RAM chips have actually become the cheapest usable option.
Had noted a correlation between RAM type and module size on FPGA boards:
   512K/1MB: QSPI
   32/64 MB: SDR SDRAM
   64/128MB: DDR1
   128MB: DDR2
   256MB: DDR3
So, maybe, this is the cheapest commodity option if they want a given RAM type.
Like, for example, when I was last looking at SDcards, 16GB was the cheapest option being sold.
Smaller sizes had fallen off the bottom, and plenty of larger sizes existed (say, 128 or 256GB).
So, even if a 4 or 8 GB SDcard would be sufficient, 16GB was what was available (in the projects I was doing, typically the biggest file on the SDcard ended up being the swapfile...).

So IMO it is highly unclear why manufacturers use large caches.
One possible explanation could be benchmarketing and using
obsolete benchmarks.  Another could be inertia with customers
thinking that "larger cache is better".
 

Cache, and RPM, probably...

Another things is fragmenting market into different "kinds" of
drives.  Rationally, high performance drives should get
better mechanical parts.  But in given performance area there
seem to be no reason for different mechanics, so I suspect
that they use the same.  They may get different firmware.
"Green" consumer parts seem to be quite aggressive powering
down (IIUC on recent WD parts it is impossible to permanently
disable this), but beyond this it is not clear to me if there
are rational reasons for significantly different firmware.
 

My experience:
   WD Green, Power-use optimized
     Drives worked pretty well, but WD seemingly phased it out for HDDs.
     WD Green is back, but mostly for SSDs.
   WD Blue, marked as general purpose;
     Not great and worse reliability IME.
     Seems to be actually the more "budget optimized" line.
   WD Black, marketed as performance optimized.
     Typically 7200RPM
     Not much notable difference IME from the WD Reds
   WD Red, optimized for NAS
     Typically 5400RPM
     Have had mostly good results with these.
   WD Purple, optimized for video usage and similar.
     Typically 5400RPM.
     No first hand experience.
Doesn't seem to be an obvious difference in cache sizes between the drive families.
There does seem to be a weak positive correlation between cache size and drive size.
RPM seems to be negatively correlated with capacity:
   10K RPM: Seemingly mostly under 1TB
   7200RPM, mostly 1TB to 4TB drives
   5400RPM, most of the bigger drives.
In my use, I had seemingly been seeing the best results from WD Red drives.
As for the specifics of what they have tuned exactly, I am not sure.
Did see claims that there are apparently no real functional differences between WD Red and WD Purple drives.
Though, apparently the WD Purple drives do have more going on in terms of corrosion resistance (possibly hydrophobic coatings or similar?, for whatever reason...). Marketing doesn't seem to say anything about being optimized for use in damp conditions though.
But, dampness and oil resistance could make sense if one had an "industrial use" optimized drive. But, in this case, would likely expect a sealed drive with something like a butyl rubber and/or PTFE coating, and possibly gold plated contacts (well, and/or a 2.5" HDD inside of a special 3.5" enclosure, made from PTFE and butyl rubber and TPU, also optimizing for shock resistance, *1).
*1: Say, if you had an HDD that was designed to be thrown around the room and subjected to other high G-force shocks, while also being periodically submerged in water, oil, and various corrosive liquids. Outer shell could be mostly PTFE, possibly then PET, butyl, then TPU, possibly with some internal heat-transfer structures (possibly a small heat pump), with gold contacts, circuits to protect from voltage transients, ... Bonus points if it can also withstand operating at high pressures (such as being sunk to the bottom of the ocean, and/or driven over by a car or large truck, ...).
Though this implies that the enclosure partly be made out of stainless steel or similar, probably still with a PTFE outer layer (where, PTFE has a stronger chemical resistance than stainless steel, but stainless steel would be needed for crush resistance).
So, layers (outer to inner): PTFE (first line chemical defense), Stainless (crush resistance), PTFE (physical damage + corrosives), butyl (shock, second line chem), TPU (shock). Then, a small heat pump between the inner HDD and stainless shell (likely primarily PTFE construction).
Well, sadly, even with all this, it would still not likely be able to operate in Venus-like conditions (if the first layer of PTFE got damaged and/or the heat pump were insufficient). Chances could maybe be improved here if one added an gold or iridium coating to the stainless steel layer, and possibly using 316 stainless (say, if the PTFE gets scratched, the stainless doesn't get corroded by all the sulfuric acid, if the coating is breached, 316 would at least corrode slower than 304, ...).
Gold contacts have decent chemical resistance. Iridium has stronger chemical and thermal resistance than gold, but is also more expensive.
While cheaper and moderately chemical resistant, lead would be insufficient for Venus like conditions. Though, in these conditions, loss of power would also ruin the drive (I would not expect the drive proper to survive direct exposure to 230C temperatures, ...).
Though, such a thing would unlikely be a mass-market device (would likely be impractically expensive).
...