Re: PCIe MSI-X interrupts

Scott Lurndal wrote:
Again allow me to express my gratitute on the quality of your posts !
A couple of dumb questions to illustrate how much more I need to
learn::

mitchalsup@aol.com (MitchAlsup1) writes:

Scott Lurndal wrote:

The PCI address (aka Stream ID) is passed to the interrupt
controller and IOMMU and used as an index to determine the
page table root pointer.

>

The stream id format is

>

    <2:0>  PCI function number
    <7:3>  PCI device number
    <15:8> PCI bus number
    <xx:16> PCI segment (root complex) number.

>
I use ChipID for the last field in case each chip has its own
PCIe tree. {Except that the bits are placed elsewhere in the address.}

Each root complex needs to be an unique segment.  A single
SRIOV endpoint can consume the entire 8-bit bus space and
the 8-bit dev/function space.  In this context, a root complex
can be considered a PCI express controller with one or more
root ports.  Each root port should be considered an unique
'segment'.

This is for device discovery, which uses the PCI express
"Extended Configuration Access Method" (aka ECAM) to scan
the PCI configuration spaces of all PCI ports.

Within a 'Chip' there are k cores, 1 last level cache, and
1 HostBridge with (say) 256 pins at its disposal. Said
pins can be handed out in powers of 2 of 4-pins each
so multiple PCIe trees of differing widths emanate from
the 256-PCIe-pins.
I guess you are calling each point of emanation a root.
I just bundle them under 1 HostBridge, and consider how
the "handing out" is done to be a HostBridge problem.
But As seen on the on-chip interconnect there is one
HostBridge which accesses all devices attached to this
Chip. Basically, I see on-chip-interconnect with one HostBridge knowing that the pins will be allocated
"efficiently" for the attached devices.
Thanks for the ECAM pointer, that clears up a raft of
questions.

>
But (now with the new CXL) instead of allocating 200+ pins
to DRAM those pins can be allocated to PCIe links; making any
chip much less dependent on which DRAM technology, which chip-
to-chip repeaters,... So, the thought is all I/O is PCIe + CXL;
and about the only other pins chip gets are RESET and ClockIn.

Note that bridging to PCI signalling will increase latency
somewhat, even with PCIe gen 6.

Unavoidable.

>
Bunches of these pins can be 'configured' into standard width
PCIe links (at least until one runs out of pins.)
>
Given that one has a PCIe root complex with around 256-pins
available, does one need multiple roots of such a wide tree ?

You basically need a root per device to accommodate SRIOV
devices (like enterprise grade network adapters, high-end
NVMe devices, etc).

As noted above: I knew more bits than B:D,F were needed,
but not which and where. And if a single SR-IOV device
consumes a whole B:D,F space sobeit. ECAM alignment identifies those bits and the routings.
I guess reading m post backwards I did not pose any questions.
My thanks again.