[openib-general] Re: [uDAPL] dat.conf generator

[James Lentini]

On Sun, 23 Apr 2006, Dotan Barak wrote:

> On Wednesday 19 April 2006 22:17, Arlin Davis wrote:
> > >>>>>>OpenIB-cma u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "mthca0 1" ""
> > >>>>>>OpenIB-cma0-1 u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "mthca0 1" ""
> > >>>>>>OpenIB-cma0-2 u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "mthca0 2" ""
> > >>>>>>OpenIB-cma1-1 u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "mthca1 1" ""
> > >>>>>>OpenIB-cma1-2 u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "mthca1 2" ""
> > >>>>>>
> > These entries are wrong. The cma versopm will only work with an ip 
> > address,  network hostname, or netdev name. The port value is 
> > meaningless since the name gives you the device and port reference all 
> > in one.
> > 
> > For cma the best flavor is netdev name as follow:
> > 
> > OpenIB-cma u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "ib0 0" ""
> > 
> > because it allows you to have identical dat.conf setups on across your 
> > cluster if you intend on using the first IB interface on each node.
> 
> but, what if one wants to work with the second ib I/F (or with the third)?
> 
> If in automatic way i will create a dat.conf that will have the following lines (on a machine with 2 HCAs, 2 port in each HCA):
> 
> OpenIB-cma u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "ib0 0" ""
> OpenIB-cma0 u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "ib0 0" ""
> OpenIB-cma1 u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "ib1 0" ""
> OpenIB-cma2 u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "ib2 0" ""
> OpenIB-cma3 u1.2 nonthreadsafe default /usr/lib/libdaplcma.so mv_dapl.1.2 "ib3 0" ""
> 
> In this file, there is a dapl provider name for every IPoIB I/F, and 
> still there is a default entry (the first I/F).
> 
> Is this is usefull?

Arlin was pointing out that the uDAPL CMA provider requires an IP 
address value, not a device name string:

OpenIB-cma u1.2 nonthreadsafe default libdapl.so of_udapl.1.2 "192.168.0.47 1" ""

The uDAPL CMA provider only uses the port value to check the device 
attributes, so the IP addr is the most important way to specify the 
device.

Naming the uDAPL providers OpenIB-cma0, OpenIB-cma1, OpenIB-cma2, ... 
makes sense, but remember that it is the IP address that will 
determine which device they map to.

Having a default provider, OpenIB-cma, is useful.