[ofa-general] [PATCH RFC v3 1/2] RDMA/Core: MEM_MGT_EXTENSIONS support

[Tom Tucker]

On Mon, 2008-05-26 at 16:02 -0700, Roland Dreier wrote:
> > The "invalidate local stag" part of a read is just a local sink side
>  > operation (ie no wire protocol change from a read).  It's not like
>  > processing an ingress send-with-inv.  It is really functionally like a
>  > read followed immediately by a fenced invalidate-local, but it doesn't
>  > stall the pipe.  So the device has to remember the read is a "with inv
>  > local stag" and invalidate the stag after the read response is placed
>  > and before the WCE is reaped by the application.
> 
> Yes, understood.  My point was just that in IB, at least in theory, one
> could just use an L_Key that doesn't have any remote permissions in the
> scatter list of an RDMA read, while in iWARP, the STag used to place an
> RDMA read response has to have remote write permission.  So RDMA read
> with invalidate makes sense for iWARP, because it gives a race-free way
> to allow an STag to be invalidated immediately after an RDMA read
> response is placed, while in IB it's simpler just to never give remote
> access at all.
> 

So I think from an NFSRDMA coding perspective it's a wash...

When creating the local data sink, We need to check the transport type.

If it's IB --> only local access,
if it's iWARP --> local + remote access.

When posting the WR, We check the fastreg capabilities bit + transport type bit:
If fastreg is true -->
	Post FastReg
	If iWARP (or with a cap bit read-with-inv-flag)
		post rdma read w/ invalidate
	else /* IB */
		post rdma read
		post invalidate
	fi
else
	... today's logic
fi	

I make the observation, however, that the transport type is now overloaded
with a set of required verbs. For iWARP's case, this means rdma-read-w-inv,
plus rdma-send-w-inv, etc... This also means that new transport types will
inherit one or the other set of verbs (IB or iWARP).

Tom


>  - R.
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general