Re: Is Parallel Programming Hard, And, If So, What Can You Do About It?

On Tue, 13 May 2025 17:49:17 -0700, Stephen Fuld wrote:

On 5/13/2025 5:18 PM, Lawrence D'Oliveiro wrote:
>

You can tell that’s wrong because the drive cache is slower than the OS
filesystem cache.

 
I don't think that proves anything.  I gave you an example of where the
drive cache speeded up the sequence of two reads to where it was faster
than two reads into the file cache.

Think of what a cache is for in the first place. The only reason they work
is because of the “principle of locality”. This can also be expressed as
saying that typical patterns of data access by application programs follow
a Pareto distribution, less formally known by monikers like the “80/20
rule” or the “90/10 rule”.

Let’s use the “90/10 rule” name just to put some concrete numbers on the
whole idea: this says that 90% of (recent) data accesses will be to just
10% of the data.

For “recent”, let’s say “within the last 10 seconds”. So the OS cache
should be big enough to hold that 10% of the data that the app accessed in
about the last 10 seconds. Of course the precise set of frequently-
accessed data (the “working set”) will drift over time. So anything beyond
this last-10-seconds’ worth will require hitting the actual disk device,
which will happen eventually.

At that point, the cache on the disk device is going to come into play.
Trouble is it’s nowhere near this kind of size: it can only hold enough
data for, say, about the last 1 second’s worth of accesses. So by the time
the OS cache experiences a miss like this, the disk cache isn’t going to
be able to help much, since it is more likely than not going to miss as
well. To be useful, it needs to be about 10 times the size of the OS
cache, so it can hold more of the 90% of the data that the app wants to
access less frequently.

But you can’t get drives like that.