[ofa-general] Problems running dtest - DAT_INVALID_ADDRESS

[small at cs.fsu.edu]

I am VERY new to infiniband and I am trying to figure out the DAT API  
so I can perhaps create a simple ping-pong program using uDAPl.  I  
have not found any good examples of accessing infiniband through a  
user API.  My adviser has a sixteen node infiniband cluster that we  
have used primarily for MPI research and  when I try and run the dtest  
program on one of the nodes I get the following error:

$ dtest
24924 Running as server - OpenIB-cma
24924: Error Adaptor open: DAT_INVALID_ADDRESS

I saw a previous thread with a similar problem and the solution was to  
add the ib1 ip address to the /etc/dat.conf file, but I cannot find  
access to any infiniband ip information.  We have ethernet adapters on  
each node which we use to address each node in MPI programs.  But then  
with the shared file system I have 16 ip addresses and one copy of  
dat.conf.  Which ip address(') should I put into it and where? When I  
do an 'ifconfig' no ib adapter is present and when I try 'ifconfig  
ib0' I get:

ib0       Link encap:UNSPEC  HWaddr   
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
           BROADCAST MULTICAST  MTU:2044  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:128
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)


When I do an ifconfig on ib1-9 they do not exist.
Here is my /etc/dat.conf :

...
#
OpenIB-cma u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so  
dapl.1.2 "ib0 0" ""
OpenIB-cma-1 u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so  
dapl.1.2 "ib1 0" ""
OpenIB-cma-2 u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so  
dapl.1.2 "ib2 0" ""
OpenIB-cma-3 u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so  
dapl.1.2 "ib3 0" ""
OpenIB-bond u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so  
dapl.1.2 "bond0 0" ""

Any help would be much appreciated though I know I am probably missing  
something very simple and obvious (or just have a too limited  
understanding of my adviser's cluster).  Also, I would like to know if  
the uDAPl is in fact the best way to interface with infiniband and any  
links to help me get started with it would be nice.


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.