[openib-general] APM support in openib stack

[Hal Rosenstock]

On Thu, 2007-01-04 at 11:31, Or Gerlitz wrote:
> On 1/4/07, Tang, Changqing <changquing.tang at hp.com> wrote:
> 
> > > >     + use IB PORT GIDs at your initial mpi init exchange
> > > >     + issue IB SA Path query via libibsa (which does not exist)
> > >
> > > likely multipath query if you want APM
> 
> > Yes, we want to have APM as well.
> 
> The RDMA CM is about to be enhanced to support IB APM where for now
> you would need to provide the alternate path from some "out of rdma
> cm" band means. But in the future an enhancemnt of the IB SM would
> allow the RDMA CM to provide you with the alternate path as well.

What enhancement are you referring to ? I think there is some APM
support available today in some SMs.

> > > There are other possibilities here too using inband (IB) exchanges.
> > > libibsa is not that far away either. What is your time frame ?
> 
> > Is it possible to have it in OFED 1.2 ?
> 
> I don't see any much advantge for you with libibsa over hard coding a
> path, ie use dynamic GIDs/LIDs and static or hacked setting for the
> other path record fields.

Other than perhaps not needing another network to transfer these
parameters.

> > > Also, you would make more work if iWARP support is of
> > > interest as well.
> > > It comes "for free" or at least "for cheap" if the RDMA CM
> > > approach is taken.
> 
> > We understand that, but we hope to have a connect/accept style IB
> > connection setup, without IPoIB involved,
> > like HP-UX IT-API(similar to uDAPL without underlying IP support), it
> > works with multiple cards.
> 
> CQ,
> 
> Please note that to have iWARP support you **must** use the rdmacm to
> establish the RDMA connection.
> 
> This is b/c unlike IB, iWARP connection can't be hard wire manually.

currently. There was a thread on perhaps changing this but it did seem
to terminate without conclusion.

-- Hal

> First a TCP connection is established and then this connection is
> moved to RDMA. Since you cann't hard wire a TCP connection you must
> use the IWARP CM etc for that. So bottom line, using the RDMA CM would
> give you IB and iWARP support and you can't support iWARP without
> it...
> 
> Or.