Re: MSI interrupts

In article <efd7a61f2d4995d42225ecf999b6786f@www.novabbs.org>,
MitchAlsup1 <mitchalsup@aol.com> wrote:

On Wed, 26 Mar 2025 23:48:17 +0000, Scott Lurndal wrote:

mitchalsup@aol.com (MitchAlsup1) writes:

On Wed, 26 Mar 2025 21:35:01 +0000, Scott Lurndal wrote:

[snip]
If you can NAK a snoop, are you then assuming that all DMA is
non-coherent?   That will really piss off the kernel folks.

>
Not at all. DMA is as coherent as the I/O MMU's translating
PTE prescribes.
>
But; why would any sane coder be performing an ATOMIC event on
memory that can be blindly overwritten by DMA ??? Or for that
mater, critical section data being overwritten (at some unknown
and unpredictable time) by DMA ??

>
Polling on a completion status which the device DMAs
(often with an allocate hint) is not uncommon.   A 32-bit
word in memory updated by the device - the allocate hint
might update the LLC directly and invalidate L1/L2 or
may update the target cache line that the driver is
polling directly.

>
Why do this ATOMICally ?? why not just read it ?? Or is this a
grab (read and modify so someone else doesn't get it.) ??

Consider multiword DMA descriptors in a ring buffer in DRAM,
shared with a device.  Manipulating these races against hardware
that's consuming the ring, and on many devices you need to do a
multiword write to transfer ownership of a descriptor to the
device for processing (say, setting both a DMA address and an
ownership bit in two consecutive words).  A few particular
models of ethernet controllers in Cortex-M7 series parts from ST
come to mind as examples of devices that do this.  Of course
proper maintenance of ring tail pointers in the device coupled
with a barrier can mitigate this, but it would be nice to do
both the descriptor and tail pointer updates atomically; instead
of maintaining a separate tail pointer into the ring, you could
just initiate a start operation by writing an arbitrary value
into a register on the device.

Indeed, for the aforementioned ST part, many drivers have gotten
this stuff wrong.  See, for example,
https://github.com/oxidecomputer/hubris/pull/1493 and
https://github.com/embassy-rs/embassy/pull/2155

- Dan C.