[openib-general] dapl broken for iWARP

[Michael Krause]

At 07:43 AM 2/8/2007, Kanevsky, Arkady wrote:
>That is correct.
>I am working with Krishna on it.
>Expect patches soon.
>
>By the way the problem is not DAPL specific
>and so is a proposed solution.
>
>There are 3 aspects of the solution.
>One is APIs. We suggest that we do not augment these.
>That is a connection requestor sets its QP
>RDMA ORD and IRD.
>When connection is established user can check the QP RDMA ORD and IRD
>to see what he has now to use over the connection.
>We may consider to extend QP attributes to support transport specific
>parameters passing in the future.
>For example, iWARP MPA CRC request.
>
>Second is the semantic that CM provides.
>The proposal is to match IBCM semantic.
>That is CM guarantee that local IRD is >= remote ORD.
>This guarantees that incoming RDMA Read requests will not overwhelm
>the QP RDMA Read capabilities.
>Again there is not changes to IBCM only to IWCM.
>Notice that as part of this IWCM will pass down to driver and extract
>from driver
>needed info.
>
>The final part is iWARP CM extension to exchange RDMA ORD, IRD.
>This is similar to IBTA Annex for IP Addressing.
>The harder part that this will eventually require IETF MPA spec extension,
>and the fact that MPA protocol is implemented in RNIC HW by many vendors,
>and hence can not be done by IWCM itself.

We looked at this quite a bit during the creation of the 
specification.   All of the targeted usage models exchange this information 
as part of their "hello" or login exchanges.    As such, the "hum" was to 
not change MPA to communicate such information and leave it to software to 
exchange these values through existing mechanisms.   I seriously doubt 
there will be much support for modifying the MPA specification at this 
stage since the implementations are largely complete and a modification 
would have to deal with the legacy interoperability issue which likely 
would be solved in software any way.  It would be simpler to simply modify 
the underlying DAPL implementation to exchange the information and keep 
this hidden from both the application and the RNIC providers.

Mike


>Thanks,
>
>Arkady Kanevsky                       email: arkady at netapp.com
>Network Appliance Inc.               phone: 781-768-5395
>1601 Trapelo Rd. - Suite 16.        Fax: 781-895-1195
>Waltham, MA 02451                   central phone: 781-768-5300
>
>
> > -----Original Message-----
> > From: Steve Wise [mailto:swise at opengridcomputing.com]
> > Sent: Wednesday, February 07, 2007 6:12 PM
> > To: Arlin Davis
> > Cc: openib-general
> > Subject: Re: [openib-general] dapl broken for iWARP
> >
> > On Wed, 2007-02-07 at 15:05 -0800, Arlin Davis wrote:
> > > Steve Wise wrote:
> > >
> > > >On Wed, 2007-02-07 at 14:02 -0600, Steve Wise wrote:
> > > >
> > > >
> > > >>Arlin,
> > > >>
> > > >>The OFED dapl code is assuming the responder_resources and
> > > >>initiator_depth passed up on a connection request event
> > are from the
> > > >>remote peer.  This doesn't happen for iWARP.  In the
> > current iWARP
> > > >>specifications, its up to the application to exchange this
> > > >>information somehow. So these are defaulting to 0 on the
> > server side
> > > >>of any dapl connection over iWARP.
> > > >>
> > > >>This is a fairly recent change, I think.  We need to come up with
> > > >>some way to deal with this for OFED 1.2 IMO.
> > > >>
> > > >>
> > > Yes, this was changed recently to sync up with the rdma_cm changes
> > > that exposed the values.
> > >
> > > >>
> > > >>
> > > >
> > > >The IWCM could set these to the device max values for instance.
> > > >
> > > >
> > > That would work fine as long as you know the remote
> > settings will be
> > > equal or better. The provider just sets the min of local device max
> > > values and the remote values provided with the request.
> > >
> >
> > I know Krishna Kumar is working on a solution for exchanging
> > this info in private data so the IWCM can "do the right
> > thing".  Stay tuned for a patch series to review for this.
> > But this functionality is definitely post OFED-1.2.
> >
> >
> > So for the OFED-1.2, I will set these to the device max in the IWCM.
> > Assuming the other side is OFED 1.2 DAPL, then it will work fine.
> >
> > Steve.
> >
> >
> >
> > _______________________________________________
> > openib-general mailing list
> > openib-general at openib.org
> > http://openib.org/mailman/listinfo/openib-general
> >
> > To unsubscribe, please visit
> > http://openib.org/mailman/listinfo/openib-general
> >
>
>_______________________________________________
>openib-general mailing list
>openib-general at openib.org
>http://openib.org/mailman/listinfo/openib-general
>
>To unsubscribe, please visit 
>http://openib.org/mailman/listinfo/openib-general