Re: Is Parallel Programming Hard, And, If So, What Can You Do About It?

On 5/23/2025 7:36 AM, MitchAlsup1 wrote:

On Fri, 23 May 2025 6:09:37 +0000, BGB wrote:
 

On 5/23/2025 12:35 AM, Lawrence D'Oliveiro wrote:

On Fri, 23 May 2025 00:18:53 -0000 (UTC), Waldek Hebisch wrote:
>

It is pretty clear that due to drive mechanics track cache/buffer is
useful.

>
Only if you don’t take the statistics of real-world cache behaviour into
account.
>

However, the real question is about size: how big should it be.

>
It can never be big enough to make a difference.

>
Sometimes, it is not the size of the cache that matters.
>
>
Say, for example, a typical cache configuration in my core:
   L1 D$: 32K, direct mapped
   L1 I$: 16K, direct mapped
   L2: 256K, direct mapped.
>
OK, so say I stick a 4K cache between the L1 and L2 caches.
Seems kinda useless just based on sizes.

 It is called a "victim buffer"
 

I had usually understood it that a "victim buffer" was typically glued directly to the L1 cache. This was outside the L1 cache, and after the main TLB.
So, the L1 ring consisted of:
   Join/VC -> L1D$ -> L1I$ -> TLB -> Join/VC
The Join/VC stage was either a joiner between the L1 ring and L2 ring, or the 4K victim cache (optional, but does help).
The L2 ring generally had the L2 cache, ROMs, a bridge to the MMIO bus, and the display and rasterizer modules (which had native interfaces on the ringbus).

Except: This small cache is 4-way set associative and so can absorb a
bunch of conflict misses, and notably reducing the number of cache
misses in the L2 cache. It can bring a performance benefit, despite
being small.

 And generally fully associative
 

On the FPGA, the cost difference between a 4-way 4K cache and 4-entry (full assoc) cache was small, but the using LUTRAMs gave a better hit-rate.
The cost difference between 4-way and 8-way was a lot more steep (and seemingly doesn't improve hit rate enough to compensate for the fairly significant cost increase).
I suspect it might be different tradeoffs on an ASIC though.

Why?

 Because it helps.

Yes.