Re: Is Parallel Programming Hard, And, If So, What Can You Do About It?

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> wrote:

On 5/25/2025 11:05 AM, Waldek Hebisch wrote:

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> wrote:

On 5/23/2025 2:03 PM, Stefan Monnier wrote:

Stephen Fuld [2025-05-23 08:28:44] wrote:

On 5/22/2025 5:18 PM, Waldek Hebisch wrote:

It is pretty clear that due to drive mechanics track cache/buffer
is useful.

Pretty clear to everyone except one person. :-)

>
🙂
>

However, the real question is about size: how big
should it be.  For "consumer" drives I see claims of 256 MB
cache.  Given rather optimistic 200 MB/s transfer rate it is
about 1.25s of drive data, that is 80-150 rotations.  I would
expect that say 4 tracks should be enough for reading.  For
writing one could use few more tracks.  Still, advertised cache
sizes seem to be much bigger than necessary.

It's not just the rotations, but the seek time. So your example is fewer
"operations" than the 80-150 you get when just including rotations.

>
I don't understand what you're getting at, here.
I think Waldek's argument is that 256MB corresponds approximately
to the amount of data stored in 80-150 tracks, and seek time doesn't
change that fact.

>
Yes, I didn't express myself well.  :-(  And once again, I have to say
that my information may be obsolete.
>
I think it is useful to separate talking about read data from write
data.  For read data, as with any cache, more is always better than
less, though with diminishing returns.  Why pick 1.25 sec as the "cut
off point"?  If the host re-references data that it hasn't read for say
3 seconds, having it in cache still saves, probably a seek time and on
average 1/2 rotation time.  Plus, it means the heads will be free to
handle other requests.  All of this is standard cache benefits.  I see
no reason to limit the cache size and reduce this benefit.

 
We are talking here about common case, that is when disc is accessed
via OS cache.  OS cache is significantly larger than disc cache, so
hit ratio for data sent to host is going to be quite low.  Disc
cache has an advantage: it gets "for free" some data that host did
not request.  But it is rather unlikely that keeping such data
for long time has significant advantage.
 

And if you are caching writes, more cache gives you more blocks to choose
from when optimizing the write back order, which reduces the time to write
them all back.

>
IIUC, for SATA drives, NCQ is still limited to 32 in-flight commands, so
unless the drive is allowed to do write-back caching it seems the amount
of space used for write-buffering is likely small (compared to 256MB).
[ Unless it is common for individual write commands to cover multi-MB
    chunks of data?  ]

>
>
For write data, I was unaware of the 32 operation limit.  I was used to
SCSI, which, IIRC was larger, and for server type applications, where
some sort of UPS is more common, the site may choose to enable write
caching in the disk.  For a disk vendor, given the small cost of the
DRAM, it is an easy choice.

 
I do not look at details of disc protocol.  But with protocal done
right host would first transfer commands and then deliver data
in order requested by the drive.  So most buffering would be in
the host and disc would need just enough buffering to ensure
smooth transmission and low interrupt rate.  4 track looks like
plenty for this purpose.

 
No, when the disk receives a write command, it accepts the write data
immediately (up to some large limit).  That way, when the heads settle
on the track, if the disk happens to be positioned in the middle of the
transfer, it can write the last part of the data to the disk
immediately, then wait for the disk to spin to where the transfer starts
to finish the transferring the first part of the write data.  This
reduces average latency, i.e. improves performance.

Latency of writes typically is of little importance, host buffers
several seconds of of write data and writes only after delay
(ratinale for this is that data may be overwritten, by dealying
host may avoid actual disc operation).  To allow scheduling in
the drive one wants commands to be sent as fast as possible.
Sending possibly bulky write data before sending next command
looks counterpoductive.  There could be some data that host
wants to be in persistent storage as fast as possible, but
making it the only option clearly would be a design error in
the disc protocol.
 

The larger DRAM is a small component of drive cost, so the
manufacturers think it is worth including more.

>
In some markets (e.g. home routers), the size of DRAM seems to be enough
of a cost factor that it took many years until reaching 256MBs, even
though those boxes *need* that RAM for all kinds of purposes (the 128MB
of my current home-router seems to be its main source of instability).
but HDDs are pretty damn expensive beasts nowadays (because prices have
not gone down for the last 10 years or so), so I guess that makes
the relative cost of 512MB of DRAM "negligible"?

>
I can't comment on routers, but for disks, while the cost of the disk
may not have come down, increasing capacity allows reduced cost per
gigabyte.  A substantial portion of the cost is not subject to Moore's
law (e.g. drive motor, magnets and arm assembly, etc.) and some capacity
increasing technologies cost more (but not enough more to overwhelm the
capacity advantage).

 
In nineties I read that for motherboard manufactures 1 cent was
"negligible", but 10 cents was significant: In volume transactions
margins were low and no party were willing to absorb 10 cents
per piece "loss".  Discs probably are less competitive than
motherboards, but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.

 
I can't comment on your specific numbers, but assuming you are right,
adding $1 to the cost is is small, at least in the part of the market I
was familiar with.  And remember, you are not "adding" 256MB, as some of
that is needed for various internal operations.

Generously, buffering could do with about 8MB.  I am not sure
how drives handle bad sector map, that potentially could be
quite large.  But in principle drive could read infor about bad
sectors from the track, keeping in RAM only info about say bad
tracks and current track.  In such case I see no reason for
large internal RAM.

--
                              Waldek Hebisch