[ofa-general] [PATCH RFC v3 1/2] RDMA/Core: MEM_MGT_EXTENSIONS support

[Or Gerlitz]

Steve Wise wrote:
> Usage Model:
> - MR made VALID and bound to a specific page list via ib_post_send(IB_WR_FAST_REG_MR)
> - MR made INVALID via ib_post_send(IB_WR_INVALIDATE_MR)
Hi Steve, Roland,

After discussing the rkey renew and fencing with send/rdma ops, I am 
quite clear with how all this plugs well into ULPs such as SCSI or FS 
low-level (interconnect) initiator/target drivers, specifically those 
who use a transactional protocol. Few more points to clarify are (sorry 
if it became somehow long):

* Do we want it to be a must for a consumer to invalidate an fast-reg mr 
before reusing it? if yes, how?

* If remote invalidation is supported, when the peer is done with the 
mr, it sends the "response" in
send-with-invalidate fashion and saves the mapper side from doing a 
local invalidate. For the case of the mapping produced by SCSI initiator 
or FS client, when remote invalidation is not supported, I don't see how 
a local invalidate design can be made in a pipe-lined manner - Since 
from the network perspective the I/O is done, the target response at 
your hands, but until doing mr invalidation the pages are typically not 
returned to the upper layer and the ULP has to "stall" till the 
invalidation WR is completed? I don't say its a bug or a big issue, just 
wonder what are your thoughts regarding this point.

* talking about remote invalidation, I understand that it requires 
support of both sides (and hence has to be negotiated), so the 
IB_DEVICE_SEND_W_INV device capability says that a device can 
send-with-invalidate, do we need a IB_DEVICE_RECV_W_INV cap as well?

* what about ZBVA, is it orthogonal to these calls, no enhancement of 
the suggested API is needed even if zbva is used, or the other way, it 
would work also when zbva is not used?

Or