Re: Is Parallel Programming Hard, And, If So, What Can You Do About It?

On 5/23/2025 12:35 AM, Lawrence D'Oliveiro wrote:

On Fri, 23 May 2025 00:18:53 -0000 (UTC), Waldek Hebisch wrote:
 

It is pretty clear that due to drive mechanics track cache/buffer is
useful.

 Only if you don’t take the statistics of real-world cache behaviour into
account.
 

However, the real question is about size: how big should it be.

 It can never be big enough to make a difference.

Sometimes, it is not the size of the cache that matters.
Say, for example, a typical cache configuration in my core:
   L1 D$: 32K, direct mapped
   L1 I$: 16K, direct mapped
   L2: 256K, direct mapped.
OK, so say I stick a 4K cache between the L1 and L2 caches.
Seems kinda useless just based on sizes.
Except: This small cache is 4-way set associative and so can absorb a bunch of conflict misses, and notably reducing the number of cache misses in the L2 cache. It can bring a performance benefit, despite being small.
Why?
Consider you have data at 0x14000000 and 0x1C00000 both being used in at the same time. The L1 cache will miss, and the L2 will miss. This small 4-way cache, well, it can absorb this miss; sparing the L2 from the miss. Likewise, in this case, the small 4-way cache is cheaper than it would be to make either the L1 or L2 caches set-associative.
So, one can't so easily just write off the HDD's cache...
You would need to prove that there is a 100% overlap between whatever the OS side cache caches, and whatever the HDD caches. But, because the OS cache is likely to be caching "clusters that were actually accessed" and the HDD cache may also cache "other stuff that happens to be in the same track, but was not requested by the host." There is not a strict 1:1 overlap.
Also, the Host/HDD interactions are not strictly sequential either, but rather because they are each operating independently over a message-passing interface, they are effectively more operating in terms of the rules of phi calculus (and so, "A is bigger and faster, B is smaller and slower" logic doesn't necessarily apply; more the relative timing and latency of the interactions over the shared communication medium).
It is like asking "why have a FIFO in a RS232 UART, why not just busy loop for every byte?". Even if the UART is slow either way (and an 8 or 16 byte FIFO is seemingly too small to matter), it may still waste less time CPU side to have a FIFO than to just have the CPU busy-loop until it can send another byte. Though, sadly, this strategy doesn't work with SPI (had to instead use 1/8/32 byte bursts).
...