Re: MSI interrupts

On 2025-03-13 2:42 p.m., Scott Lurndal wrote:

Robert Finch <robfi680@gmail.com> writes:

On 2025-03-13 12:34 p.m., MitchAlsup1 wrote:

On Thu, 13 Mar 2025 4:50:47 +0000, Robert Finch wrote:
>

Planning on having the Q+ CPU respond directly to MSI interrupts. My
thought is to integrate an interrupt controller directly into the CPU
which will have several queues to prioritize interrupts. The controller
will take care of detecting the highest priority interrupt to process,
performing priority inversion occasionally, and detect stuck interrupts.

>
My 66000 went all the way to MSI-X interrupts as its base.
>
There are mapping methods to map INTx and INTk into MSI-X messages.
MSI->MSI-X mapping is "just zero extend the message".

>
MSI interrupts are the way to go when there are transistors available.
Just having discovered them not to long ago, I still think they are great.
I have the luxury of not needing to support legacy interrupt systems
(yet), all the peripheral cores needing interrupts will have to support
MSI responses.

 MSI has been deprecated for years.  MSI-X is preferred and used by
most modern devices (and supported by major operating software).
 

>
The Interrupt port lives in the memory controller and takes an
aperture of addresses and maps them to various interrupt tables.
>

I have read that MSI devices target particular I/O addresses to send the
messages to. Not sure what having a dedicated I/O addresses buys. In my
architecture, rather than target an I/O address a specific processing
core number is specified.

 There are many reasons that MSI-X supports vector specific target addresses.
  1) It is possible to target DRAM (so software can poll for the interrupt)

This can be done using a dummy core in the system that echos interrupts messages back to the DRAM. A current issue with the way interrupts in my system are working, is they always spit out the interrupt on the response bus, which means they cannot act like the master.
To drive messages onto the bus as requests instead of responses every peripheral device would need to be able to be a bus master. It would add a lot of logic to the system. Extra cross-bar ports etc.

  2) It is possible to support level-sensitive semantics by using two vectors;
     one to assert the interrupt and the second to deassert it.  Each needs
     to target a different interrupt controller register (address).

I think this can be handled with the cause code being returned as part of the response.

  3) A system may have multiple interrupt controllers for different
     security states (e.g. monitor/secure vs. hypervisor vs. direct virtual
     to a guest).

Is this just to get more target vectors for each interrupt? Could there be one interrupt controller with multiple target vectors depending on the security state?
Q+ CPU always switches to machine mode to process interrupts. It can then redirect to other operating modes. A security state code could be added to the response so that different vector tables may be used.

 

>
My architecture goes with a 3-bit priority field, and an 8-bit field to
help resolve the cause of the interrupt. The bus/device/function is also
part of the interrupt response.

 That's what ARM64 GIC does, each interrupt is accompanied by a 20+ bit
'device id', which is composed of the segment, bus, dev/func values
for the sending device.   Since each PCIe controller can consume the
entire bus space potentially, the segment widens the device id space
to include a large number of devices.   The 'device id' is used to
select the in-memory table used to map the interrupt to the target
guest and the target guest vector number.

Seems like going out to memory tables to fetch the targets of an interrupt would be costly, perhaps slow. In the SoC there are already a half dozen devices fighting for access to the DRAM.

 I think you really want an interrupt controller as an intermediary
between the device and the CPU interrupt signal, allows a great deal
of implementation flexibility (e.g. changing the target CPU in the

Absolutely. Be crazy not to. There is an irq controller component separate from the CPU. (I decided to move the controller outside of the CPU).

interrupt controller tables rather than requiring the device driver
to change the msix-vector address field).
 

Updating I/O is faster than updating DRAM tables.
I need to study more on the use of interrupt tables.