[ofa-general] [RFP] support for iWARP requirement - activeconnectside MUST send first FPDU

[Felix Marti]

> -----Original Message-----
> From: Tom Tucker [mailto:tom at opengridcomputing.com]
> Sent: Tuesday, October 23, 2007 9:32 PM
> To: Felix Marti
> Cc: Kanevsky, Arkady; Glenn Grundstrom; Sean Hefty; Steve Wise; Roland
> Dreier; interop-wg at lists.openfabrics.org; OpenFabrics General
> Subject: Re: [ofa-general] [RFP] support for iWARP requirement -
> activeconnectside MUST send first FPDU
> 
> Felix Marti wrote:
> >
> >> -----Original Message-----
> >> From: general-bounces at lists.openfabrics.org [mailto:general-
> >> bounces at lists.openfabrics.org] On Behalf Of Kanevsky, Arkady
> >> Sent: Tuesday, October 23, 2007 6:26 PM
> >> To: Glenn Grundstrom; Sean Hefty; Steve Wise
> >> Cc: Roland Dreier; interop-wg at lists.openfabrics.org; OpenFabrics
> >> General
> >> Subject: RE: [ofa-general] [RFP] support for iWARP requirement -
> >> activeconnectside MUST send first FPDU
> >>
> >> This is still a protocol and should be defined by IETF not OFA.
> >> But if we get agreement from all iWARP vendors this will be a good
> >> step.
> >>
> > [felix] This will not work with a Chelsio RNIC which follows the
IETF
> > specification by a) not issuing a 0B RDMA Write to the wire and b)
> > silently consuming an incoming 0B write. Therefore 0B RDMA Writes
> cannot
> > be 'abused' for such a synchronization mechanism. I believe that the
> > mentioned apps adhering to the iWarp requirement do a 'send' from
the
> > active side and only have the passive side issue RDMA ops once the
> > incoming send has been received. I would guess that following a
> similar
> > model is the best way to go and supported by all iWarp vendors
> > implementing the IETF spec.
> >
> >
> IMO, the iWARP vendors _must_ get together and work on MPA '2'.
> Standardizing FPDU 'abuse' might be a good place to start, but it
needs
> to be fixed to support peer-to-peer going forward.
> 
> In the mean-time, imperfectly hiding the issue in the Firmware, uDAPL,
> the iWARP CM or anywhere else except the application seems to me to be
> the only customer friendly solution.

[felix] While I'm not against trying to hide the connection migration
details somewhere below the ULP, I'm not convinced that the issue is as
severe as you make it to be and I would not press to have the issue
resolved in a matter that requires a new MPA version. In fact, the
different rdma transports (and maybe even different versions of the same
transport (in the case of IB)) provide different features and I would
assume that ULPs will eventually code to these features and must thus be
aware of the underlying transport protocol. In that bigger picture, the
connection migration issue at hand seems fairly trivial to solve even if
it requires an ULP change... 
 
> >
> >> If we can not get agreement on it on reflector lets do
> >> it at SC'07 OFA dev. conference.
> >>
> >> Arkady Kanevsky                       email: arkady at netapp.com
> >> Network Appliance Inc.               phone: 781-768-5395
> >> 1601 Trapelo Rd. - Suite 16.        Fax: 781-895-1195
> >> Waltham, MA 02451                   central phone: 781-768-5300
> >>
> >>
> >>
> >>> -----Original Message-----
> >>> From: Glenn Grundstrom [mailto:ggrundstrom at NetEffect.com]
> >>> Sent: Tuesday, October 23, 2007 9:02 PM
> >>> To: Sean Hefty; Steve Wise
> >>> Cc: Roland Dreier; interop-wg at lists.openfabrics.org;
> >>> OpenFabrics General
> >>> Subject: RE: [ofa-general] [RFP] support for iWARP
> >>> requirement - activeconnect side MUST send first FPDU
> >>>
> >>>
> >>>>> That is what I've been trying to push.  Both MVAPICH2 and
> >>>>>
> >>>> OMPI have been
> >>>>
> >>>>> open to adjusting their transports to adhere to this
> >>>>>
> > requirement.
> >
> >>>>> I wouldn't mind implementing something to enforce this in
> >>>>>
> >>>> the IWCM or
> >>>>
> >>>>> the iWARP drivers IF there was a clean way to do it.  So
> >>>>>
> >>> far there
> >>>
> >>>>> hasn't been a clean way proposed.
> >>>>>
> >>>> Why can't either uDAPL or iW CM always do a send from the active
> >>>>
> > to
> >
> >>>> passive side that gets stripped off?  From the active side,
> >>>>
> >>> the first
> >>>
> >>>> send is always posted before any user sends, and if
> >>>>
> >>> necessary, a user
> >>>
> >>>> send can be queued by software to avoid a QP/CQ overrun.  The
> >>>> completion can simply be eaten by software.  On the passive
> >>>>
> >>> side, you
> >>>
> >>>> have a similar process for receiving the data.
> >>>>
> >>> This is similar to an option in the NetEffect driver.  A zero
> >>> byte RDMA write is sent from the active side and accounted
> >>> for on the passive side.  This can be turned on and off by
> >>> compile and module options for compatibility.
> >>>
> >>> I second Sean's question - why can't uDAPL or the iw_cm do this?
> >>>
> >>>
> >>>> (Yes this adds wire protocol, which requires both sides to
support
> >>>> it.)
> >>>>
> >>>> - Sean
> >>>>
> >>>>
> >>> _______________________________________________
> >>> general mailing list
> >>> general at lists.openfabrics.org
> >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>>
> >>> To unsubscribe, please visit
> >>> http://openib.org/mailman/listinfo/openib-general
> >>>
> >>>
> >> _______________________________________________
> >> general mailing list
> >> general at lists.openfabrics.org
> >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>
> >> To unsubscribe, please visit
> >>
> > http://openib.org/mailman/listinfo/openib-
> >
> >> general
> >>
> > _______________________________________________
> > general mailing list
> > general at lists.openfabrics.org
> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >
> > To unsubscribe, please visit
> http://openib.org/mailman/listinfo/openib-general
> >