Re: MSI interrupts

On 2025-03-15 1:46 p.m., MitchAlsup1 wrote:

On Sat, 15 Mar 2025 2:10:48 +0000, Robert Finch wrote:
 

Okay, all of this MSI-X stuff is starting to percolate into my brain.
Coming up with a lighter-weight version of MSI-X for Q+. MSI-X has more
capacity than is needed for the Q+ SoC. I desire something in between
the simple and the complex. Interrupts are complex in multi-master
systems.

 Interrupts should utilize a message pathway already used by other
"system things". This eliminates any cost "ON" the interrupts
themselves.
>

Agree. Basically, I am using a “cheezy” approach to try and reduce the amount of logic. It would be better done with bus master ports on slave device and a point-to-point interconnect.
In Q+ SoC the bus slave response pathway is being reused to signal interrupts. Rather than force peripheral devices to contain both a bus master and a bus slave interface, most peripherals contain only the slave interface (uart, keyboard).
The Q+ CPU currently only acts a bus master, not a slave device, hence a dedicated IRQ vector bus to the CPU. <- I may change this to a generic slave bus interface, but a bus master interface would also need to be added to the interrupt controller. It would be more hardware.
I am trying to avoid “doubling up” on the system bus. It is currently a tree routed from the CPU core to slave devices. It is not a point-to-point interconnection. Other devices (frame buffer) needing bus master capability are doing so through the memory controller to the DRAM. They do not have access to I/O.

Secondly, you want to be in a position where is someone chooses Q+
as their main CPUs, that they are not hindered in building large
systems using it.
>

Yikes. I had not thought of someone choosing Q+. I do not have experience building large hardware systems. Undoubtedly, there would be all kinds of pitfalls with it. I have been using Q+ as a learning tool.

Q+ SoC has just 32-bits for the IRQ message payload data as many devices
are 32-bit and IRQ messages are passed on the 32-bit response bus. That
means a full I/O address cannot be used as it would be too large. As the
message data, the I/O device supplies the vector number (12-bits) and
interrupt controller number (6 bits). Specific interrupt controllers are
targeted rather than an I/O address as targeting requires only six bits
rather than a 32+ bit I/O address. Currently there will be only one
interrupt controller in the system. Interrupt controllers could be
designed to do other things like log to memory.

 Under advice of council (EricP), I punted HW vectoring--which removed
a couple of pointers from the device:function control block along with
restrictions on where said table is, and on what alignment it is.
>

I thought the address of the hardware table would have to be specified in a device control register for MSI-X. The address for triggering the interrupt would be a location in the table. Are not the low order bits of this address the same as the interrupt vector number? I had split this location into two pieces (vector number, plus int controller number) so that a 32-bit register could specify the (64-bit) memory address.
I added an IMSI controller. This allows a bus master to generate interrupts easily with software. It basically reflects the interrupt request onto the slave response bus. The interrupt request is generated by writing to a memory address.

Each interrupt controller has a base addresses and length for the vector
tables so they may be located in system RAM (DRAM). The interrupt
controller takes care of fetching the vector and broadcasting the vector
to a specific CPU core.

 There are situations where you want any 1 from a set of cores to "take"
the interrupt, rather than targeting one core (which may be busy at the
time.
 

I have more work to do on this. I don’t know how to approach it if there is more than a small number of cores. For the number of cores fitting into the FPGA, I suppose the six-bit target core number could just be an index into a table containing a list of cores that are targeted. With 64 CPU cores for instance, each entry in the table would be a 64-bit vector.
The SoC system is limited to 64 CPU cores by the use of a 6-bit core number. So, a 64-bit vector containing which cores to send the interrupt to could be used. A find-first-one responding could be used to mask things so that only one CPU sees the interrupt.

                        The target CPU core is programmable in the
vector table. The vector contains 128-bits of information (almost same
format as MSI-X) and identifies the ISR address and processing core.
32-bits of data may also be supplied in the vector table entry.
>
To simplify things, the current interrupt controller contains the vector
table in BRAM in a compressed format which supports 512 vectors for each
operating mode. (2048 vectors total). The vector table is mapped into
the MMI/O space. The interrupt controller reads from its internal vector
table and directly supplies the CPU with the vector on a dedicated bus.

 Those kinds of details have been punted to SW.
 

I had the thought of supplying an instruction to the CPU rather than an
address, ala 8080. Normally this would be a JMP instruction to the ISR.
Other instructions may be useful like task-switch or interrupt return.