[ofa-general] [RFP] support for iWARP requirement - activeconnectside MUST send first FPDU
Tom Tucker
tom at opengridcomputing.com
Wed Oct 24 07:40:11 PDT 2007
Felix Marti wrote:
>
>> -----Original Message-----
>> From: Tom Tucker [mailto:tom at opengridcomputing.com]
>> Sent: Tuesday, October 23, 2007 9:32 PM
>> To: Felix Marti
>> Cc: Kanevsky, Arkady; Glenn Grundstrom; Sean Hefty; Steve Wise; Roland
>> Dreier; interop-wg at lists.openfabrics.org; OpenFabrics General
>> Subject: Re: [ofa-general] [RFP] support for iWARP requirement -
>> activeconnectside MUST send first FPDU
>>
>> Felix Marti wrote:
>>
>>>> -----Original Message-----
>>>> From: general-bounces at lists.openfabrics.org [mailto:general-
>>>> bounces at lists.openfabrics.org] On Behalf Of Kanevsky, Arkady
>>>> Sent: Tuesday, October 23, 2007 6:26 PM
>>>> To: Glenn Grundstrom; Sean Hefty; Steve Wise
>>>> Cc: Roland Dreier; interop-wg at lists.openfabrics.org; OpenFabrics
>>>> General
>>>> Subject: RE: [ofa-general] [RFP] support for iWARP requirement -
>>>> activeconnectside MUST send first FPDU
>>>>
>>>> This is still a protocol and should be defined by IETF not OFA.
>>>> But if we get agreement from all iWARP vendors this will be a good
>>>> step.
>>>>
>>>>
>>> [felix] This will not work with a Chelsio RNIC which follows the
>>>
> IETF
>
>>> specification by a) not issuing a 0B RDMA Write to the wire and b)
>>> silently consuming an incoming 0B write. Therefore 0B RDMA Writes
>>>
>> cannot
>>
>>> be 'abused' for such a synchronization mechanism. I believe that the
>>> mentioned apps adhering to the iWarp requirement do a 'send' from
>>>
> the
>
>>> active side and only have the passive side issue RDMA ops once the
>>> incoming send has been received. I would guess that following a
>>>
>> similar
>>
>>> model is the best way to go and supported by all iWarp vendors
>>> implementing the IETF spec.
>>>
>>>
>>>
>> IMO, the iWARP vendors _must_ get together and work on MPA '2'.
>> Standardizing FPDU 'abuse' might be a good place to start, but it
>>
> needs
>
>> to be fixed to support peer-to-peer going forward.
>>
>> In the mean-time, imperfectly hiding the issue in the Firmware, uDAPL,
>> the iWARP CM or anywhere else except the application seems to me to be
>> the only customer friendly solution.
>>
>
> [felix] While I'm not against trying to hide the connection migration
> details somewhere below the ULP, I'm not convinced that the issue is as
> severe as you make it to be and I would not press to have the issue
> resolved in a matter that requires a new MPA version. In fact, the
> different rdma transports (and maybe even different versions of the same
> transport (in the case of IB)) provide different features and I would
> assume that ULPs will eventually code to these features and must thus be
> aware of the underlying transport protocol. In that bigger picture, the
> connection migration issue at hand seems fairly trivial to solve even if
> it requires an ULP change...
>
I didn't make an argument about severity. Qualifying the severity is in
the customer's purview. I'm simply pointing out the following: a) the
perspective that the restriction is trivial is how we got here, b)
making the app change is putting a decision in the customer's hands that
IMO an iWARP vendor would rather they didn't have to make "Do I or don't
I support iWARP?", and c) you have the power to hide this behavior for
most cases.
Finally, I believe RFC means "Request for Comment". Well here's one last
comment -- "Add an FPDU message at the end of MPA exchange and fix the
problem in the protocol."
>
>
>>>> If we can not get agreement on it on reflector lets do
>>>> it at SC'07 OFA dev. conference.
>>>>
>>>> Arkady Kanevsky email: arkady at netapp.com
>>>> Network Appliance Inc. phone: 781-768-5395
>>>> 1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195
>>>> Waltham, MA 02451 central phone: 781-768-5300
>>>>
>>>>
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Glenn Grundstrom [mailto:ggrundstrom at NetEffect.com]
>>>>> Sent: Tuesday, October 23, 2007 9:02 PM
>>>>> To: Sean Hefty; Steve Wise
>>>>> Cc: Roland Dreier; interop-wg at lists.openfabrics.org;
>>>>> OpenFabrics General
>>>>> Subject: RE: [ofa-general] [RFP] support for iWARP
>>>>> requirement - activeconnect side MUST send first FPDU
>>>>>
>>>>>
>>>>>
>>>>>>> That is what I've been trying to push. Both MVAPICH2 and
>>>>>>>
>>>>>>>
>>>>>> OMPI have been
>>>>>>
>>>>>>
>>>>>>> open to adjusting their transports to adhere to this
>>>>>>>
>>>>>>>
>>> requirement.
>>>
>>>
>>>>>>> I wouldn't mind implementing something to enforce this in
>>>>>>>
>>>>>>>
>>>>>> the IWCM or
>>>>>>
>>>>>>
>>>>>>> the iWARP drivers IF there was a clean way to do it. So
>>>>>>>
>>>>>>>
>>>>> far there
>>>>>
>>>>>
>>>>>>> hasn't been a clean way proposed.
>>>>>>>
>>>>>>>
>>>>>> Why can't either uDAPL or iW CM always do a send from the active
>>>>>>
>>>>>>
>>> to
>>>
>>>
>>>>>> passive side that gets stripped off? From the active side,
>>>>>>
>>>>>>
>>>>> the first
>>>>>
>>>>>
>>>>>> send is always posted before any user sends, and if
>>>>>>
>>>>>>
>>>>> necessary, a user
>>>>>
>>>>>
>>>>>> send can be queued by software to avoid a QP/CQ overrun. The
>>>>>> completion can simply be eaten by software. On the passive
>>>>>>
>>>>>>
>>>>> side, you
>>>>>
>>>>>
>>>>>> have a similar process for receiving the data.
>>>>>>
>>>>>>
>>>>> This is similar to an option in the NetEffect driver. A zero
>>>>> byte RDMA write is sent from the active side and accounted
>>>>> for on the passive side. This can be turned on and off by
>>>>> compile and module options for compatibility.
>>>>>
>>>>> I second Sean's question - why can't uDAPL or the iw_cm do this?
>>>>>
>>>>>
>>>>>
>>>>>> (Yes this adds wire protocol, which requires both sides to
>>>>>>
> support
>
>>>>>> it.)
>>>>>>
>>>>>> - Sean
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> general mailing list
>>>>> general at lists.openfabrics.org
>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>>>>
>>>>> To unsubscribe, please visit
>>>>> http://openib.org/mailman/listinfo/openib-general
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> general mailing list
>>>> general at lists.openfabrics.org
>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>>>
>>>> To unsubscribe, please visit
>>>>
>>>>
>>> http://openib.org/mailman/listinfo/openib-
>>>
>>>
>>>> general
>>>>
>>>>
>>> _______________________________________________
>>> general mailing list
>>> general at lists.openfabrics.org
>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>>
>>> To unsubscribe, please visit
>>>
>> http://openib.org/mailman/listinfo/openib-general
>>
More information about the general
mailing list