Re: Is Parallel Programming Hard, And, If So, What Can You Do About It?

On Sun, 18 May 2025 1:35:54 +0000, Lawrence D'Oliveiro wrote:

On Tue, 13 May 2025 17:49:17 -0700, Stephen Fuld wrote:
>

On 5/13/2025 5:18 PM, Lawrence D'Oliveiro wrote:
>

You can tell that’s wrong because the drive cache is slower than the OS
filesystem cache.

>
I don't think that proves anything.  I gave you an example of where the
drive cache speeded up the sequence of two reads to where it was faster
than two reads into the file cache.

>
Think of what a cache is for in the first place. The only reason they
work
is because of the “principle of locality”. This can also be expressed as
saying that typical patterns of data access by application programs
follow
a Pareto distribution, less formally known by monikers like the “80/20
rule” or the “90/10 rule”.
>
Let’s use the “90/10 rule” name just to put some concrete numbers on the
whole idea: this says that 90% of (recent) data accesses will be to just
10% of the data.
>
For “recent”, let’s say “within the last 10 seconds”.

Principle is reasonable, specified time is not.
A 4MB cache can be filled in 1ms when memory performs at (only) 4GB/s
(and the memory systems are up in the 40GB/s peak territory while
caches remain rather small (several GB at most).)

So the OS cache
should be big enough to hold that 10% of the data that the app accessed
in
about the last 10 seconds. Of course the precise set of frequently-
accessed data (the “working set”) will drift over time. So anything
beyond
this last-10-seconds’ worth will require hitting the actual disk device,
which will happen eventually.

Just change 10 seconds to 10 milliseconds and I have nothing to complain
about.

At that point, the cache on the disk device is going to come into play.
Trouble is it’s nowhere near this kind of size: it can only hold enough
data for, say, about the last 1 second’s worth of accesses. So by the
time
the OS cache experiences a miss like this, the disk cache isn’t going to
be able to help much, since it is more likely than not going to miss as
well. To be useful, it needs to be about 10 times the size of the OS
cache, so it can hold more of the 90% of the data that the app wants to
access less frequently.
>
But you can’t get drives like that.