Re: Stacks, was Segments

On Fri, 14 Feb 2025 19:51:44 +0000, Scott Lurndal wrote:

mitchalsup@aol.com (MitchAlsup1) writes:

-------------

Update:
>
I have figured out how to re-attach Guest Physical BAR back
as MMI/O commands enter the top of a PCIe tree.
>
Thanks to Scott Lurndal for being gentle with me.

>
Here is an example topology from a Raptor Lake system:
>
  bus:dev.function    (bus 0 is a traditional PCI bus)
  Region X is BAR .
>
This devices are all built-in to either the core or
the PCH/southbridge.
>
The first plugin-pci card would reside on bus 4.
>
Only intel systems provide or use the I/O port (legacy 8086) BARs.
>

It is going to take me some time to dig through this.

>
$ lspci -vvv | egrep "^[0-9]|Region "
00:00.0 Host bridge: Intel Corporation Raptor Lake-S 8+12 - Host
Bridge/DRAM Controller (rev 01)
00:02.0 VGA compatible controller: Intel Corporation Raptor Lake-S GT1
[UHD Graphics 770] (rev 04) (prog-if 00 [VGA controller])
        Region 0: Memory at 6000000000 (64-bit, non-prefetchable)
[size=16M]
        Region 2: Memory at 4000000000 (64-bit, prefetchable)
[size=256M]
        Region 4: I/O ports at 5000 [size=64]
00:04.0 Signal processing controller: Intel Corporation Raptor Lake
Dynamic Platform and Thermal Framework Processor Participant (rev 01)
        Region 0: Memory at 6001100000 (64-bit, non-prefetchable)
[size=128K]
00:08.0 System peripheral: Intel Corporation GNA Scoring Accelerator
module (rev 01)
        Region 0: Memory at 600113b000 (64-bit, non-prefetchable)
[disabled] [size=4K]
00:14.0 USB controller: Intel Corporation Alder Lake-S PCH USB 3.2 Gen
2x2 XHCI Controller (rev 11) (prog-if 30 [XHCI])
        Region 0: Memory at 6001120000 (64-bit, non-prefetchable)
[size=64K]
00:14.2 RAM memory: Intel Corporation Alder Lake-S PCH Shared SRAM (rev
11)
        Region 0: Memory at 6001134000 (64-bit, non-prefetchable)
[disabled] [size=16K]
        Region 2: Memory at 600113a000 (64-bit, non-prefetchable)
[disabled] [size=4K]
00:16.0 Communication controller: Intel Corporation Alder Lake-S PCH
HECI Controller #1 (rev 11)
        Region 0: Memory at 6001139000 (64-bit, non-prefetchable)
[size=4K]
00:17.0 SATA controller: Intel Corporation Alder Lake-S PCH SATA
Controller [AHCI Mode] (rev 11) (prog-if 01 [AHCI 1.0])
        Region 0: Memory at 70700000 (32-bit, non-prefetchable)
[size=8K]
        Region 1: Memory at 70704000 (32-bit, non-prefetchable)
[size=256]
        Region 2: I/O ports at 5080 [size=8]
        Region 3: I/O ports at 5088 [size=4]
        Region 4: I/O ports at 5060 [size=32]
        Region 5: Memory at 70703000 (32-bit, non-prefetchable)
[size=2K]
00:1a.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root
Port #25 (rev 11) (prog-if 00 [Normal decode])
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root
Port #3 (rev 11) (prog-if 00 [Normal decode])
00:1c.3 PCI bridge: Intel Corporation Device 7abb (rev 11) (prog-if 00
[Normal decode])
00:1f.0 ISA bridge: Intel Corporation Device 7a86 (rev 11)
00:1f.3 Audio device: Intel Corporation Alder Lake-S HD Audio Controller
(rev 11)
        Region 0: Memory at 6001130000 (64-bit, non-prefetchable)
[size=16K]
        Region 4: Memory at 6001000000 (64-bit, non-prefetchable)
[size=1M]
00:1f.4 SMBus: Intel Corporation Alder Lake-S PCH SMBus Controller (rev
11)
        Region 0: Memory at 6001138000 (64-bit, non-prefetchable)
[size=256]
        Region 4: I/O ports at efa0 [size=32]
00:1f.5 Serial bus controller: Intel Corporation Alder Lake-S PCH SPI
Controller (rev 11)
        Region 0: Memory at 70702000 (32-bit, non-prefetchable)
[size=4K]
01:00.0 Non-Volatile memory controller: Sandisk Corp WD PC SN5000S M.2
2230 NVMe SSD (DRAM-less) (prog-if 02 [NVM Express])
        Region 0: Memory at 70600000 (64-bit, non-prefetchable)
[size=16K]
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet Controller (rev 1b)
        Region 0: I/O ports at 4000 [size=256]
        Region 2: Memory at 70504000 (64-bit, non-prefetchable) [size=4
        Region 4: Memory at 70500000 (64-bit, non-prefetchable)
[size=16K]
03:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8852BE
PCIe 802.11ax Wireless Network Controller
        Region 0: I/O ports at 3000 [size=256]
        Region 2: Memory at 70400000 (64-bit, non-prefetchable)
[size=1M]
>
The traditional PCI bus supports a 5-bit device# and a 3-bit function #.
A PCIe bus supports either a 3-bit function number and the high-order
five bits MBZ (i.e. only device 0 is legal on PCIe) or, if the function
advertises
the Alternate Routing Identifier (ARI) capability, an 8-bit function
number
(SRIOV leverages ARI to support dense routing IDs, but any bus that
supports
ARI can handle 256 physical functions.
>
PCI supports two forms of configuration space addresses in TLPs:
   Type 0 contains only a function number and the register address.
   Type 1 contains the bus number, function number and register address.

PCIe segments go where ? Are they "picked off" prior to being routed
down the tree ??

A PCI-PCI bridge (such as the root complex port bridge) will translate
a type 1 transaction to type 0 when the target RID is on the
configured secondary bus, or forward the type 1 transaction to
a subordinate bus bridge.   With ARI, the upstream bridge
from the endpoint needs to be configured as ARI enabled so that
it forwards type 1 transactions to the secondary bus, as the
SRIOV Routing IDs can extend into the 8-bit bus space (allowing
up to 65535 virtual functions associated with a single PF).