[ofa-general] Re: iWARP peer-to-peer CM proposal

[Kanevsky, Arkady]

Agree with initiator/client sending signalled 0B RDMA Read.
This will handle client side.

Still not 100% clear on passive/server side.
Two issues which bothers me.
1. Is "bogus" S-tag allowed for incomming RDMA ops?
I do not recall that RDDP requires that length is checked before
S-tag.

2. How is "verb" layer on server side knows that RDMA Read op
came and was done? Is it some back door to vendor FW?
Will this be kicked for all incoming RDMA Read ops?

Arkady Kanevsky                       email: arkady at netapp.com
Network Appliance Inc.               phone: 781-768-5395
1601 Trapelo Rd. - Suite 16.        Fax: 781-895-1195
Waltham, MA 02451                   central phone: 781-768-5300
 

> -----Original Message-----
> From: Steve Wise [mailto:swise at opengridcomputing.com] 
> Sent: Tuesday, November 27, 2007 7:48 PM
> To: Caitlin Bestler
> Cc: Kanevsky, Arkady; Glenn Grundstrom; Leonid Grossman; 
> openib-general at openib.org
> Subject: Re: [ofa-general] Re: iWARP peer-to-peer CM proposal
> 
> Caitlin Bestler wrote:
> > On Nov 27, 2007 3:58 PM, Steve Wise 
> <swise at opengridcomputing.com> wrote:
> > 
> >> For the short term, I claim we just implement this as part 
> of linux 
> >> iwarp connection setup (mandating a 0B read be sent from 
> the active 
> >> side).  Your proposal to add meta-data to the private data 
> requires a 
> >> standards change anyway and is, IMO, the 2nd phase of this whole 
> >> enchilada...
> >>
> >> Steve.
> >>
> > 
> > I don't see how you can have any solution here that does 
> not require meta-data.
> > For non-peer-to-peer connections neither a zero length RDMA Read or 
> > Write should be sent. An extraneous RDMA Read is 
> particularly onerous 
> > for a short lived connection that fits the classic active/passive 
> > model. So *something* is telling the CMA layer that this 
> connection may need an MPA unjam action.
> > If that isn't meta-data, what is it?
> 
> I assumed the 0B read would _always_ be sent as part of 
> establishing an iWARP connection using linux and the rdma-cm.
> 
> > 
> > Further, the RDMA Read solution is adequate whenever the RDMA Write 
> > solution would have been (although at an unnecessary extra 
> cost), but 
> > as near as I can determine it is not a complete solution. If the 
> > passive side needs an untagged message completion then *something* 
> > needs to send it. How can the CM layer (or, I suppose, the 
> ULP itself) 
> > know that this untagged NOP message must be sent without meta-data?
> 
> I believe at Reno we had the current rnic vendors all saying 
> a SEND or 0B read will work.  So:  If someone has current 
> iwarp HW that will _not_
>   handle this problem by doing the 0B read hack, please speak up now.
> 
> > 
> > As I see it, if we want to do the minimum that is required, but be 
> > certain that it is adequate, we need a per-connection setup 
> meta-data exchange.
> 
> Are you going to prototype this?
> 
> 
> Steve.
> 
> 
>