Re: MSI interrupts

On Thu, 13 Mar 2025 17:49:35 +0000, Robert Finch wrote:

On 2025-03-13 12:34 p.m., MitchAlsup1 wrote:

On Thu, 13 Mar 2025 4:50:47 +0000, Robert Finch wrote:
>

Planning on having the Q+ CPU respond directly to MSI interrupts. My
thought is to integrate an interrupt controller directly into the CPU
which will have several queues to prioritize interrupts. The controller
will take care of detecting the highest priority interrupt to process,
performing priority inversion occasionally, and detect stuck interrupts.

>
My 66000 went all the way to MSI-X interrupts as its base.
>
There are mapping methods to map INTx and INTk into MSI-X messages.
MSI->MSI-X mapping is "just zero extend the message".

>
MSI interrupts are the way to go when there are transistors available.
Just having discovered them not to long ago, I still think they are
great.
I have the luxury of not needing to support legacy interrupt systems
(yet), all the peripheral cores needing interrupts will have to support
MSI responses.

>
The Interrupt port lives in the memory controller and takes an
aperture of addresses and maps them to various interrupt tables.
>

I have read that MSI devices target particular I/O addresses to send the
messages to. Not sure what having a dedicated I/O addresses buys. In my
architecture, rather than target an I/O address a specific processing
core number is specified.

MXI and MSI-X target an MMI/O address aperture (i.e., range), the
position and the meaning of the address bits and the message bits
are system dependent.
It is expected that some address bits will indicate what interrupt
table to access.
It is expected that either (or both) bits from the address and message
will indicate priority of the interrupt.
In a classical Operating System, each GuestOS gets its own interrupt
table, and any core running that GuestOS will be watching that inter-
rupt table. Hypervisors get their own interrupt tables, as do Secure
monitors. {{To hand waving accuracy}}

>

There is a broadcast mechanism that notifies cores of new higher
priority level interrupts pending, and a protocol for getting
an interrupt and notifying the IC that it will be handled by a
core.
>
Along with the 32-bit message, the architecture provides for
6-bits of priority, 2-bits of SW-stack indexing, and an 8-bit
ISR handler index number.

>
My architecture goes with a 3-bit priority field, and an 8-bit field to
help resolve the cause of the interrupt.

Having an advocate of such a position (EricP BTW) that 8-priority
levels are sufficient; made me give great through to this space.
I ended up with 64 priority levels, mainly, because I know of no
general purpose way of comparing priority levels found in JoeOS1
versus those in JoeOS2 or JoeOS3. 8-levles is arguably sufficient
for JoeOS[k]. But what if one wants some of JoeOS2's priorities
lower than all JoeOS1's priorities, but the others above them ?!?
So, 64-levels, occupying 1-bit each fit exactly the typical HW
storage container (64-bits) allows for 8-JoeOSs to have their
8-levels each to be fit into one concept we call system-priority
without any constraint on any JoeOS-chosen at random.

                                       The bus/device/function is also
part of the interrupt response. A 6-bit target core number field is
included in the response too. A target core number of all ones is
treated as a broadcast to all cores. The entire interrupt response
message is available to the CPU so software can decide how to handle the
interrupt. There is some potential for data to be passed back. The
contents of the data field passed back on the bus is

                                                     PCIe

                                                     unspecified. The
other info is passed back in the address field of the bus.

>
Interrupt tables remain active even when the virtual machine
it inactive. When next scheduled, priority ISRs will be serviced
prior to low priority thread instructions.
>
{And some things I can't talk about}

Shucks.

>

Setting up an MSI interrupt protocol for Q+.
>
The controller will sit on the CPU's main response collect bus.
Interrupt messages contain the interrupter's bus/device/function + a
cause code. They may also contain a target core number.
>
Planning on distributing the interrupt control to peripheral devices
instead of having a system level interrupt controller.

>
I sketched up an interrupt controller in SystemVerilog and it was only
about 700 LUTs although it does use seven BRAMs for priority queues.
The priority queues can hold hundreds of interrupt messages which is
likely far more than would be needed. So if a queue overflows it would
indicate an issue with the interrupt system. The controller also keeps a
message history with timestamps. If the same message is present at the
start and end of the history along with a short difference in the
timestamp, it is considered a stuck interrupt.
>
The controller inverts interrupt priorities for one cycle every 64 clock
cycles. The priority inversion is with respect to which queue is read
from. The idea is to try and prevent a low-priority task that is waiting
on an interrupt from being permanently disabled.