[ewg] RE: [ofa-general] how do I use uDAPL with iWARP?

[Scott Weitzenkamp (sweitzen)]

I tried that, and it didn't work:

[root at svbu-qa2950-1 ~]# grep eth /etc/dat.conf
OpenIB-cma u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "eth2 0"
""
[root at svbu-qa2950-1 ~]# dtest
10194 Running as server - OpenIB-cma
10194 Error dat_ep_create: DAT_INVALID_HANDLE
10194 Error freeing EP: DAT_INVALID_HANDLE DAT_INVALID_HANDLE_EP

10194: DAPL Test Complete.

10194: Message RTT: Total=      0.00 usec, 10 bursts, itime=      0.00
usec, pc=
0
10194: RDMA write:  Total=      0.00 usec, 10 bursts, itime=      0.00
usec, pc=
0
10194: RDMA read:   Total=      0.00 usec,   4 bursts, itime=      0.00
usec, pc
=0
10194: RDMA read:   Total=      0.00 usec,   4 bursts, itime=      0.00
usec, pc
=0
10194: RDMA read:   Total=      0.00 usec,   4 bursts, itime=      0.00
usec, pc
=0
10194: RDMA read:   Total=      0.00 usec,   4 bursts, itime=      0.00
usec, pc
=0
10194: open:        32254.93 usec
10194: close:       31936.17 usec
10194: PZ create:       7.15 usec
10194: PZ free:         4.05 usec
10194: LMR create:     36.00 usec
10194: LMR free:       22.89 usec
10194: EVD create:      6.91 usec
10194: EVD free:       11.92 usec
10194: EP create:      28.85 usec
10194: EP free:         0.00 usec
10194: TOTAL:         106.57 usec

Scott Weitzenkamp
SQA and Release Manager
Data Center Access Engineering
Cisco Systems


 

> -----Original Message-----
> From: Joshua Bernstein [mailto:jbernstein at penguincomputing.com] 
> Sent: Wednesday, April 02, 2008 4:05 PM
> To: Scott Weitzenkamp (sweitzen)
> Cc: [ofa_general]; OpenFabrics EWG
> Subject: Re: [ofa-general] how do I use uDAPL with iWARP?
> 
> Scott,
> 
> On Apr 2, 2008, at 4:00 PM, Scott Weitzenkamp (sweitzen) wrote:
> > I have OFED 1.3 and a Chelsio S310E-SR+ iWARP 10GE NIC.  I have  
> > ib_rdma_lat working, so I know IB verbs are working.
> >
> > How do I use uDAPL, though?  All the default /etc/dat.conf entries  
> > have IPoIB or bonding interfaces in them.
> 
> What you will want to do is edit /etc/ofed/dat64.conf or other  
> related dat.conf file and change the name of the device from 
> "ib0" to  
> the name of the interface that the Chelsio card came up as. So for  
> example with my NetXen cards coming up at eth2, so for example the  
> first two lines of my /etc/ofed/dat64.conf file look like this:
> 
> OpenIB-cma u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so dapl. 
> 1.2 "eth2 0" ""
> #OpenIB-cma u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so dapl. 
> 1.2 "ib0 0" ""
> 
> Notice how I've commented out the ib0 line and simply changed 
> that to  
> be eth2. Then you can use say HP-MPI for example using the -UDAPL  
> option. Other MPI stacks have similar methods of telling them to use  
> the UDAPL transport.
> 
> -Joshua Bernstein
> Software Engineer
> Penguin Computing
>