[ofa-general] Re: iWARP peer-to-peer CM proposal

[Steve Wise]

Caitlin Bestler wrote:
> 
>> -----Original Message-----
>> From: Steve Wise [mailto:swise at opengridcomputing.com]
>> Sent: Tuesday, November 27, 2007 4:48 PM
>> To: Caitlin Bestler
>> Cc: Kanevsky, Arkady; Glenn Grundstrom; Leonid Grossman; openib-
>> general at openib.org
>> Subject: Re: [ofa-general] Re: iWARP peer-to-peer CM proposal
>>
>> Caitlin Bestler wrote:
>>> On Nov 27, 2007 3:58 PM, Steve Wise <swise at opengridcomputing.com>
>> wrote:
>>>> For the short term, I claim we just implement this as part of linux
>>>> iwarp connection setup (mandating a 0B read be sent from the active
>>>> side).  Your proposal to add meta-data to the private data requires
>> a
>>>> standards change anyway and is, IMO, the 2nd phase of this whole
>>>> enchilada...
>>>>
>>>> Steve.
>>>>
>>> I don't see how you can have any solution here that does not require
>> meta-data.
>>> For non-peer-to-peer connections neither a zero length RDMA Read or
>> Write
>>> should be sent. An extraneous RDMA Read is particularly onerous for a
>> short
>>> lived connection that fits the classic active/passive model. So
>> *something*
>>> is telling the CMA layer that this connection may need an MPA unjam
>> action.
>>> If that isn't meta-data, what is it?
>> I assumed the 0B read would _always_ be sent as part of establishing an
>> iWARP connection using linux and the rdma-cm.
>>
> 
> That is an extra round-trip per connection setup, which is a significant
> penalty for a short lived connection. It is trivial for HPC/peer-to-peer
> applications, but would be a killer for something like HTTP over RDMA.
> 
> Doing something like this for *every* connection makes it effectively
> a change to the MPA protocol. OFA is not the forum for such discussions,
> the IETF is.
> 
> OFA drafting an understanding of how peer-to-peer applications use the
> existing protocol, on the other hand, is quite reasonable. But it has
> to be something done by peer-to-peer middleware or by the verbs layer
> in response to a flag from the peer-to-peer middleware. Otherwise it
> is not augmenting a protocol, it is changing it.
> 

posting a 0B read after the mpa setup isn't changing the MPA protocol. 
Its adding a protocol on top of the MPA setup in order to meet the 
requirements of the MPA protocol.  Whether you add a private-data 
request for this or _assume_ the 0B read will happen doesn't change this.