Re: MSI interrupts

On 3/26/2025 1:22 PM, Chris M. Thomasson wrote:

On 3/26/2025 7:57 AM, Dan Cross wrote:

In article <jwv4izfncov.fsf-monnier+comp.arch@gnu.org>,
Stefan Monnier  <monnier@iro.umontreal.ca> wrote:

Leaving aside the unchecked boundary conditions I highlighted
above, there are other questions that this raises.  You
mentioned that the ESM stuff can use up to 8 cache lines in a
single atomic "event"; let's count how many this code may
encounter:
>
1. `from` may point to a cache line
2. `to` may point to a different cache line.
3. Does your architecture use a stack?  What sort of alignment
    criteria do you impose?   At first blush, it seems to me that
    the pointers `from_next`, `from_prev`, and `to_next` could be
    on the stack and if so, will be on at least one cache line,
    and possibly 2, if the call frame for `MoveElement` spans
    more than one.  Of course, it's impossible to say when this
    is compiled, but maybe you require that the stack is always
    aligned to a cache line boundary or something.  Or maybe
    these are all in registers and it's fine.

>
AFAIK the stack accesses will use normal loads that won't be tracked for
conflicts, so they don't count towards the 8 cache-lines limit.

>
Ok, good to go.
>

4. Depending on the size and alignment criteria of the the
    `Element` structure and the placement of the `next` and
    `prev` elements in the struct (unknowable from this example),
    `to->next` may be on another cache line.

>
The coder may need to be careful with such slot placement and
alignment, indeed.

>
Yup.  This could easily happen.  Consider,
>
#include <stdalign.h>
#include <stdint.h>
#include <stdio.h>
>
struct elem {
         struct elem *next;
         struct elem *prev;
         uint32_t data;
};

[...]
 Indeed. Ideally, elem should be padded up to _and_ aligned on a l2 cache line boundary to help get rid of false sharing. Also, the "anchor" should be padded and aligned as well. An anchor can be:
 struct anchor
{
     std::atomic<elem*> m_head;
};
 

Actually, iirc pad and align the anchor to a reservation granule boundary, then pad and align its nodes to l2 cache line boundaries.