[ofa-general] [RFP] support for iWARP requirement- activeconnectside MUST send first FPDU
Kanevsky, Arkady
Arkady.Kanevsky at netapp.com
Wed Oct 24 07:52:40 PDT 2007
The bottom line we need to single solution which works for all vendors.
This issue cause interoperability problems.
So Customers will stay on the sideline until these type of issues are
resolved.
Hiding behind protocol holes is not going to help adoption.
Will sending 0-size send message from initiator side work?
Can IWCM on responder side squeeze 0-size buffer to recv this message
and swallow it. Hope that there is no check that need to be done
on all comletions? Will work for both interrupt and polling mode?
I still believe that it will be simplier to add it to MPA protocol.
Thanks,
Arkady Kanevsky email: arkady at netapp.com
Network Appliance Inc. phone: 781-768-5395
1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195
Waltham, MA 02451 central phone: 781-768-5300
> -----Original Message-----
> From: Tom Tucker [mailto:tom at opengridcomputing.com]
> Sent: Wednesday, October 24, 2007 10:40 AM
> To: Felix Marti
> Cc: Kanevsky, Arkady; Roland Dreier; Glenn Grundstrom;
> OpenFabrics General; interop-wg at lists.openfabrics.org
> Subject: Re: [ofa-general] [RFP] support for iWARP
> requirement- activeconnectside MUST send first FPDU
>
> Felix Marti wrote:
> >
> >> -----Original Message-----
> >> From: Tom Tucker [mailto:tom at opengridcomputing.com]
> >> Sent: Tuesday, October 23, 2007 9:32 PM
> >> To: Felix Marti
> >> Cc: Kanevsky, Arkady; Glenn Grundstrom; Sean Hefty; Steve Wise;
> >> Roland Dreier; interop-wg at lists.openfabrics.org;
> OpenFabrics General
> >> Subject: Re: [ofa-general] [RFP] support for iWARP requirement -
> >> activeconnectside MUST send first FPDU
> >>
> >> Felix Marti wrote:
> >>
> >>>> -----Original Message-----
> >>>> From: general-bounces at lists.openfabrics.org [mailto:general-
> >>>> bounces at lists.openfabrics.org] On Behalf Of Kanevsky, Arkady
> >>>> Sent: Tuesday, October 23, 2007 6:26 PM
> >>>> To: Glenn Grundstrom; Sean Hefty; Steve Wise
> >>>> Cc: Roland Dreier; interop-wg at lists.openfabrics.org; OpenFabrics
> >>>> General
> >>>> Subject: RE: [ofa-general] [RFP] support for iWARP requirement -
> >>>> activeconnectside MUST send first FPDU
> >>>>
> >>>> This is still a protocol and should be defined by IETF not OFA.
> >>>> But if we get agreement from all iWARP vendors this will
> be a good
> >>>> step.
> >>>>
> >>>>
> >>> [felix] This will not work with a Chelsio RNIC which follows the
> >>>
> > IETF
> >
> >>> specification by a) not issuing a 0B RDMA Write to the
> wire and b)
> >>> silently consuming an incoming 0B write. Therefore 0B RDMA Writes
> >>>
> >> cannot
> >>
> >>> be 'abused' for such a synchronization mechanism. I
> believe that the
> >>> mentioned apps adhering to the iWarp requirement do a 'send' from
> >>>
> > the
> >
> >>> active side and only have the passive side issue RDMA ops
> once the
> >>> incoming send has been received. I would guess that following a
> >>>
> >> similar
> >>
> >>> model is the best way to go and supported by all iWarp vendors
> >>> implementing the IETF spec.
> >>>
> >>>
> >>>
> >> IMO, the iWARP vendors _must_ get together and work on MPA '2'.
> >> Standardizing FPDU 'abuse' might be a good place to start, but it
> >>
> > needs
> >
> >> to be fixed to support peer-to-peer going forward.
> >>
> >> In the mean-time, imperfectly hiding the issue in the Firmware,
> >> uDAPL, the iWARP CM or anywhere else except the
> application seems to
> >> me to be the only customer friendly solution.
> >>
> >
> > [felix] While I'm not against trying to hide the connection
> migration
> > details somewhere below the ULP, I'm not convinced that the
> issue is
> > as severe as you make it to be and I would not press to
> have the issue
> > resolved in a matter that requires a new MPA version. In fact, the
> > different rdma transports (and maybe even different versions of the
> > same transport (in the case of IB)) provide different
> features and I
> > would assume that ULPs will eventually code to these
> features and must
> > thus be aware of the underlying transport protocol. In that bigger
> > picture, the connection migration issue at hand seems
> fairly trivial
> > to solve even if it requires an ULP change...
> >
> I didn't make an argument about severity. Qualifying the
> severity is in the customer's purview. I'm simply pointing
> out the following: a) the perspective that the restriction is
> trivial is how we got here, b) making the app change is
> putting a decision in the customer's hands that IMO an iWARP
> vendor would rather they didn't have to make "Do I or don't I
> support iWARP?", and c) you have the power to hide this
> behavior for most cases.
>
> Finally, I believe RFC means "Request for Comment". Well
> here's one last comment -- "Add an FPDU message at the end of
> MPA exchange and fix the problem in the protocol."
>
> >
> >
> >>>> If we can not get agreement on it on reflector lets do
> it at SC'07
> >>>> OFA dev. conference.
> >>>>
> >>>> Arkady Kanevsky email: arkady at netapp.com
> >>>> Network Appliance Inc. phone: 781-768-5395
> >>>> 1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195
> >>>> Waltham, MA 02451 central phone: 781-768-5300
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Glenn Grundstrom [mailto:ggrundstrom at NetEffect.com]
> >>>>> Sent: Tuesday, October 23, 2007 9:02 PM
> >>>>> To: Sean Hefty; Steve Wise
> >>>>> Cc: Roland Dreier; interop-wg at lists.openfabrics.org;
> OpenFabrics
> >>>>> General
> >>>>> Subject: RE: [ofa-general] [RFP] support for iWARP
> requirement -
> >>>>> activeconnect side MUST send first FPDU
> >>>>>
> >>>>>
> >>>>>
> >>>>>>> That is what I've been trying to push. Both MVAPICH2 and
> >>>>>>>
> >>>>>>>
> >>>>>> OMPI have been
> >>>>>>
> >>>>>>
> >>>>>>> open to adjusting their transports to adhere to this
> >>>>>>>
> >>>>>>>
> >>> requirement.
> >>>
> >>>
> >>>>>>> I wouldn't mind implementing something to enforce this in
> >>>>>>>
> >>>>>>>
> >>>>>> the IWCM or
> >>>>>>
> >>>>>>
> >>>>>>> the iWARP drivers IF there was a clean way to do it. So
> >>>>>>>
> >>>>>>>
> >>>>> far there
> >>>>>
> >>>>>
> >>>>>>> hasn't been a clean way proposed.
> >>>>>>>
> >>>>>>>
> >>>>>> Why can't either uDAPL or iW CM always do a send from
> the active
> >>>>>>
> >>>>>>
> >>> to
> >>>
> >>>
> >>>>>> passive side that gets stripped off? From the active side,
> >>>>>>
> >>>>>>
> >>>>> the first
> >>>>>
> >>>>>
> >>>>>> send is always posted before any user sends, and if
> >>>>>>
> >>>>>>
> >>>>> necessary, a user
> >>>>>
> >>>>>
> >>>>>> send can be queued by software to avoid a QP/CQ overrun. The
> >>>>>> completion can simply be eaten by software. On the passive
> >>>>>>
> >>>>>>
> >>>>> side, you
> >>>>>
> >>>>>
> >>>>>> have a similar process for receiving the data.
> >>>>>>
> >>>>>>
> >>>>> This is similar to an option in the NetEffect driver.
> A zero byte
> >>>>> RDMA write is sent from the active side and accounted
> for on the
> >>>>> passive side. This can be turned on and off by compile
> and module
> >>>>> options for compatibility.
> >>>>>
> >>>>> I second Sean's question - why can't uDAPL or the iw_cm do this?
> >>>>>
> >>>>>
> >>>>>
> >>>>>> (Yes this adds wire protocol, which requires both sides to
> >>>>>>
> > support
> >
> >>>>>> it.)
> >>>>>>
> >>>>>> - Sean
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> _______________________________________________
> >>>>> general mailing list
> >>>>> general at lists.openfabrics.org
> >>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>>>>
> >>>>> To unsubscribe, please visit
> >>>>> http://openib.org/mailman/listinfo/openib-general
> >>>>>
> >>>>>
> >>>>>
> >>>> _______________________________________________
> >>>> general mailing list
> >>>> general at lists.openfabrics.org
> >>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>>>
> >>>> To unsubscribe, please visit
> >>>>
> >>>>
> >>> http://openib.org/mailman/listinfo/openib-
> >>>
> >>>
> >>>> general
> >>>>
> >>>>
> >>> _______________________________________________
> >>> general mailing list
> >>> general at lists.openfabrics.org
> >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>>
> >>> To unsubscribe, please visit
> >>>
> >> http://openib.org/mailman/listinfo/openib-general
> >>
>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit
> http://openib.org/mailman/listinfo/openib-general
>
More information about the general
mailing list