[ofa-general] how do I use uDAPL with iWARP?
jbernstein at penguincomputing.com
Wed Apr 2 16:09:10 PDT 2008
On Apr 2, 2008, at 4:07 PM, Scott Weitzenkamp (sweitzen) wrote:
> I tried that, and it didn't work:
> [root at svbu-qa2950-1 ~]# grep eth /etc/dat.conf
> OpenIB-cma u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2
> "eth2 0"
> [root at svbu-qa2950-1 ~]# dtest
> 10194 Running as server - OpenIB-cma
> 10194 Error dat_ep_create: DAT_INVALID_HANDLE
> 10194 Error freeing EP: DAT_INVALID_HANDLE DAT_INVALID_HANDLE_EP
Ah, it is using the correct device then. Do you have the rdma_ucm
>> -----Original Message-----
>> From: Joshua Bernstein [mailto:jbernstein at penguincomputing.com]
>> Sent: Wednesday, April 02, 2008 4:05 PM
>> To: Scott Weitzenkamp (sweitzen)
>> Cc: [ofa_general]; OpenFabrics EWG
>> Subject: Re: [ofa-general] how do I use uDAPL with iWARP?
>> On Apr 2, 2008, at 4:00 PM, Scott Weitzenkamp (sweitzen) wrote:
>>> I have OFED 1.3 and a Chelsio S310E-SR+ iWARP 10GE NIC. I have
>>> ib_rdma_lat working, so I know IB verbs are working.
>>> How do I use uDAPL, though? All the default /etc/dat.conf entries
>>> have IPoIB or bonding interfaces in them.
>> What you will want to do is edit /etc/ofed/dat64.conf or other
>> related dat.conf file and change the name of the device from
>> "ib0" to
>> the name of the interface that the Chelsio card came up as. So for
>> example with my NetXen cards coming up at eth2, so for example the
>> first two lines of my /etc/ofed/dat64.conf file look like this:
>> OpenIB-cma u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so dapl.
>> 1.2 "eth2 0" ""
>> #OpenIB-cma u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so dapl.
>> 1.2 "ib0 0" ""
>> Notice how I've commented out the ib0 line and simply changed
>> that to
>> be eth2. Then you can use say HP-MPI for example using the -UDAPL
>> option. Other MPI stacks have similar methods of telling them to use
>> the UDAPL transport.
>> -Joshua Bernstein
>> Software Engineer
>> Penguin Computing
More information about the general