[ofa-general] [PATCH RFC v3 1/2] RDMA/Core: MEM_MGT_EXTENSIONS support

[Steve Wise]

Roland Dreier wrote:
>  > BTW:  a single capability bit doesn't allow apps to decide at run time
>  > whether to use read-with-inv, which is iwarp-only.  Perhaps we need
>  > that as its own capbility bit?  Or perhaps we can load detailed
>  > support/no support into the query device logic?  What it some devices
>  > can only support part of the suite of MEM_MGT_EXTENSIONS?
> 
> I think RDMA read with invalidate can be tested for as iWARP vs. IB.
> The reason IB doesn't have it is kind of inherent in the IB protocol,
> since remote access is not required for the RDMA target.
> 

The "invalidate local stag" part of a read is just a local sink side 
operation (ie no wire protocol change from a read).  It's not like 
processing an ingress send-with-inv.  It is really functionally like a 
read followed immediately by a fenced invalidate-local, but it doesn't 
stall the pipe.  So the device has to remember the read is a "with inv 
local stag" and invalidate the stag after the read response is placed 
and before the WCE is reaped by the application.

> I think making the capability flags really fine-grained isn't worth
> it -- we went too far in that direction historically, and no one checks
> any capability flags at all.  It's just complexity.
> 

Ok.

Steve.