[ewg] Re: [ofa-general] how do I use uDAPL with iWARP?

[Joshua Bernstein]

On Apr 2, 2008, at 4:07 PM, Scott Weitzenkamp (sweitzen) wrote:
> I tried that, and it didn't work:
>
> [root at svbu-qa2950-1 ~]# grep eth /etc/dat.conf
> OpenIB-cma u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2  
> "eth2 0"
> ""
> [root at svbu-qa2950-1 ~]# dtest
> 10194 Running as server - OpenIB-cma
> 10194 Error dat_ep_create: DAT_INVALID_HANDLE
> 10194 Error freeing EP: DAT_INVALID_HANDLE DAT_INVALID_HANDLE_EP

Ah, it is using the correct device then. Do you have the rdma_ucm  
modules loaded?

-Josh

>> -----Original Message-----
>> From: Joshua Bernstein [mailto:jbernstein at penguincomputing.com]
>> Sent: Wednesday, April 02, 2008 4:05 PM
>> To: Scott Weitzenkamp (sweitzen)
>> Cc: [ofa_general]; OpenFabrics EWG
>> Subject: Re: [ofa-general] how do I use uDAPL with iWARP?
>>
>> Scott,
>>
>> On Apr 2, 2008, at 4:00 PM, Scott Weitzenkamp (sweitzen) wrote:
>>> I have OFED 1.3 and a Chelsio S310E-SR+ iWARP 10GE NIC.  I have
>>> ib_rdma_lat working, so I know IB verbs are working.
>>>
>>> How do I use uDAPL, though?  All the default /etc/dat.conf entries
>>> have IPoIB or bonding interfaces in them.
>>
>> What you will want to do is edit /etc/ofed/dat64.conf or other
>> related dat.conf file and change the name of the device from
>> "ib0" to
>> the name of the interface that the Chelsio card came up as. So for
>> example with my NetXen cards coming up at eth2, so for example the
>> first two lines of my /etc/ofed/dat64.conf file look like this:
>>
>> OpenIB-cma u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so dapl.
>> 1.2 "eth2 0" ""
>> #OpenIB-cma u1.2 nonthreadsafe default /usr/lib64/libdaplcma.so dapl.
>> 1.2 "ib0 0" ""
>>
>> Notice how I've commented out the ib0 line and simply changed
>> that to
>> be eth2. Then you can use say HP-MPI for example using the -UDAPL
>> option. Other MPI stacks have similar methods of telling them to use
>> the UDAPL transport.
>>
>> -Joshua Bernstein
>> Software Engineer
>> Penguin Computing
>>