Re: MSI interrupts

On Thu, 13 Mar 2025 21:14:08 +0000, Scott Lurndal wrote:

mitchalsup@aol.com (MitchAlsup1) writes:

On Thu, 13 Mar 2025 18:34:32 +0000, Scott Lurndal wrote:
>

>

Most modern devices advertise the MSI-X capability instead.

>
And why not:: its just a few more flip-flops and almost no more
sequencing logic.

>
I've seen several devices with more than 200 MSI-X vectors;
thats 96 bits per vector to store a full 64-bit address and
32-bit data payload.

At this point, with 200+ entries, flip-flops are not recommended,
instead these would be placed in a RAM of some sort. Since RAMs
come in 1KB and 2KB quanta; we have 1 of 2K and 1 of 1K and we
have 256 said message containers, with 1 cycle access (after
you get to that corner of some chip).

Keep in mind that a guest operating system may be writing
the 64-bit address field in a virtual function assigned to
it with guest virtual addresses, so the inbound path for
MSI-X needs to traverse the IOMMU before hitting the
interrupt controller logic.

An important concept that is hard to find in the PCIe specifications.

                             The guest OS also specifies
the data payload, which may be an IRQ number on an intel
system, or a virtual IRQ number translated by the IOMMU
or Interrupt controller into a physical IRQ number (allowing
multiple guest OS to use the same IRQs they would use on real
hardware).

Just note that GuestOS[k].IRQ[j] uses the same bit patterns
at the device, they get delivered to their own unique
GuestOS through its interrupt table.
But even when GuestOS[k] and GuestOS[j] use the same 96-bit MSI-X
pattern, each GuestOS can interpret its message the way it wants,
sort its drivers in the way it wants, and operate with blissful
ignorance of GuestOS[other].