[openib-general] APM support in openib stack

[Hal Rosenstock]

On Thu, 2007-01-04 at 09:30, Or Gerlitz wrote:
> Tang, Changqing wrote:
> > Sorry, I find the function  'ib_sa_path_rec_get()'  in kernel code. Then
> > here is my question:
> > 
> > Is there any way (instruction) to fill in struct 'ib_sa_path_rec' inside
> > struct 'ib_cm_req_param' without using librdmacm.so ?
> 
> Hi CQ,
> 
> I understand that you considering to go on an approach which does not 
> involve librdmacm, so you would probably like to
> 
> 	+ use IB PORT GIDs at your initial mpi init exchange
> 	+ issue IB SA Path query via libibsa (which does not exist)

likely multipath query if you want APM

> 	+ establish IB RC connection (listen/connect/accept) via libibcm
> 
> Please note that such an approach is possible even with the non 
> existence of libibsa (similarly to what IB MPIs do today use IB PORT 
> LIDs [note you would need both GIDs & LIDs to have the CM working fine] 

There are other possibilities here too using inband (IB) exchanges.
libibsa is not that far away either. What is your time frame ?

> at your ranks pre exchange and hardcode the other IB PATH params such as 
> MTU, PKEY and SL you set later into the IB RC QP.
> 
> However, a no rdma cm approach means you need to apply hacks to "guess" 
> the correct pkey, mtu and sl and some more limitations that eventually 
> you would face when coming to advanced IB deployment environments.

Also, you would make more work if iWARP support is of interest as well.
It comes "for free" or at least "for cheap" if the RDMA CM approach is
taken.

-- Hal

> All you have to do to use the rdma cm to require a functional IPoIB NIC 
> on each of the active IB PORTs, which is a trivial requirement from the 
> users in the Ethernet world, so why not apply it here as well ???
> 
> On the other hand moving to use the IB CM instead of emulating it via 
> TCP is some progress...
> 
> Or.
> 
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>