[ofa-general] [PATCH RFC v3 1/2] RDMA/Core: MEM_MGT_EXTENSIONS support

[Roland Dreier]

 > send-with-invalidate is a little different in the sense that we would
 > probably want to expose remote invalidation through libibverbs such
 > that user space block/file targets (eg the iSER layer of STGT) would
 > be able to use it, but (at least in this point of time) not expose the
 > other memory management extensions to user space.

Why?  Local invalidate and RDMA read with invalidate make perfect sense
for userspace too.  Of course fast register through a send queue can't
be used in userspace because it operations on physical memory, but I
think MEM_MGT_EXTENSIONS makes sense as something userspace can test for
and use.

 > BTW - what's the status of the send-with-invalidate patches to the
 > core and mlx4?

I'll add the completion struct changes for 2.6.27, and roll the mlx4
patches into the full MEM_MGT_EXTENSIONS patch.

 > >  ZBVA would require adding some flag to request ZBVA when registering.

 > So this flag would be added as a field in the WR? for the current
 > proposal, can the ULP dictate the VA as done with the current FMR API
 > exposed by the core?

The IB spec has a table that shows exactly which operations would need
flags to handle the ZBVA extension.  As for the VA, I think the latest
patch is pretty clear:

 > @@ -676,6 +683,20 @@ struct ib_send_wr {
 >  			u16	pkey_index; /* valid for GSI only */
 >  			u8	port_num;   /* valid for DR SMPs on switch only */
 >  		} ud;
 > +		struct {
 > +			u64				iova_start;

 - R.