Re: MSI interrupts

On 2025-03-17 9:38 a.m., Scott Lurndal wrote:

Robert Finch <robfi680@gmail.com> writes:

>

   <please trim posts>
 

>
Consider that you have a pool of 4 cores setup to receive interrupts.
That those 4 cores are running at differing priorities and the interrupt
is at a still different priority. You want the core operating at the
lowest
priority (with the right software stack) to accept the interrupt !!

>
Okay, I wrote a naive hardware filter for this which should work okay
for small numbers of CPUs but does not scale up very well.
What happens if there is no core ready? Place the IRQ back into the
queue (regenerate the IRQ as an IRQ miss IRQ)? Or just wait for an
available core. I do not like the idea of waiting as it could stall the
system.

 Use a fifo to hold up to N pending IRQ. Define a signal that asserts when
the fifo is non-empty.  The CPU can mask the signal to prevent
interruption, when the signal is unmasked the CPU pops the
first IRQ from the FIFO.   Or use a bitmap in flops or SRAM
(prioritization happensin the logic that asserts the fact
hat an interrupt is pending to the CPU).
 

There is already a queue holding N pending IRQ. It is actually P queues sorted by priority. The highest priority queue with an output (non-empty) is presented as the current IRQ. The signal is masked with an threshold value.

Choose whether you want a FIFO/bitmap per target CPU, or global
pending data with the logic to select highest priority pending
IRQ when the CPU signals that interrups are not masked.

Okay, there is one controller (FIFO queue) for up to 63 cores. There is a bitmap of which CPUs to notify. This is an affinity mask.
I was not sure what to do if none of the CPUs were ready for an IRQ. But I see now there is nothing to do as it is the highest priority interrupt that is output already. There should be no need to feed it back to the queue. It is just a matter of waiting until a core is ready. IE. there cannot be another IRQ to process as there must have been a ready core. IF there was a core able to process a lower priority IRQ then it could certainly have processed the higher priority one. So interrupts are stalled and continue to queue for that group of cores until someone can process the IRQ.
Been reading up on the RISCV AIA.
For RISCV the CPU core handling the interrupt is indirectly specified by the I/O address set in the device as there is an IMSIC for each RISCV hart. So only a single IMSIC would be responding at that address. It would be up to software to set the I/O address to pick a particular CPU core. It seems to me that MSI interrupt handling using AIA would be heavy-weight. Having a controller for every hart could be a lot of logic if there are a lot of harts. There may be more than one hart per CPU.
Reading that the I/O addresses need to be translated by an IOMMU. In Q+ since an interrupt controller number is being used instead of an I/O address there is no need to translate the address. Software needs to determine which group of cores will be handling the interrupt, and select an interrupt controller appropriately.
The Q+ controller looks a bit like an IMSIC controller as it has pending and enable bits. I note that there is an enable bit in the vector table and an enable bit array. So, I assume both bits must be enabled for an interrupt to occur.