[openib-general] SDP Protocol Issue Was: Configuration of sdp
Sean Hubbell
shubbell at dbresearch.net
Wed Nov 29 08:42:07 PST 2006
Ok, after futher inverstigation, here is what I have found:
socket failed: Address family not supported by protocol
I am assuming when sdp "exchanges" the AF_INET sockets to AF_SDP sockets
that the current code that I am running has a issue or I don't know how
to configure things...
I have running libsdp v. 0.9.0 x86_64 as the rpm from downloaded from
mirror.centos.org running kernel 2.6.9-42.0.3.plus.c4smp
Also, my /etc/libsdp.conf file has the following:
log min-level 1 destination syslog
#use both server * *:*
#use both client * *:*
use sdp client * 10.10.0.0/16:*
use sdp server * 10.10.0.0/16:*
and seems to break all of my TCP connections, regardless if the
interface is 10.10.* Is my configuration correct?
What would you recommend my next step to be? Upgrading my kernel as well
as libsdp to the latest? Also, currently without using sdp I am getting
941 MBps, what do you get when you run using sdp, Karun mentioned that
he is getting double the performance, which is where I am at; needing
better performance from my apps?
Sean
Eitan Zahavi wrote:
> The error code is what the standard connect() returns.
> Just "man connect"
>
> EZ
>
> Eitan Zahavi
> Senior Engineering Director, Software Architect
> Mellanox Technologies LTD
> Tel:+972-4-9097208
> Fax:+972-4-9593245
> P.O. Box 586 Yokneam 20692 ISRAEL
>
>
>
>> -----Original Message-----
>> From: Sean Hubbell [mailto:shubbell at dbresearch.net]
>> Sent: Wednesday, November 29, 2006 4:34 PM
>> To: Eitan Zahavi
>> Cc: Karun Sharma; openib-general at openib.org
>> Subject: Re: [openib-general] Configuration of sdp
>>
>> This is what I got in /var/log/messages:
>>
>> Nov 29 08:32:14 neptune kernel: sdp_sock(32768:0): rdma_resolve_addr
>> failed: -99
>> Nov 29 08:32:14 neptune iperf: iperf[28484] libsdp Error connect:
>>
> failed with
>
>> code <-1> for SDP fd:5
>>
>> Where can I find out about the error messages? Do I still need to go
>>
> through
>
>> the code to find these?
>>
>> Sean
>>
>> Eitan Zahavi wrote:
>>
>>> Hi Sean
>>>
>>> Regarding libsdp.conf:
>>>
>>> The simplest config that would catch all applications/ports and will
>>> NOT break your Eth based connections is:
>>>
>>> log min-level 7 destination syslog
>>> use both server * *:*
>>> use both client * *:*
>>>
>>> Not the level=7 on the "log" directive means that you will get a
>>>
> line
>
>>> in /var/log/messages for every listen/accept/connect with info that
>>> should let you know if the connection was made through SDP or TCP.
>>>
>>> Sean Hubbell wrote:
>>>
>>>> Yes, I tried that in our testing when I loaded the ib_sdp (I did
>>>>
> just
>
>>>> use the defaults). I was wondering if the numbers that we are
>>>>
> seeing
>
>>>> seem consistent with everyone elses... and at what point will the
>>>>
> sdp
>
>>>> module increase my bandwidth when using a legacy / third party
>>>> network library?
>>>>
>>>> Sean
>>>>
>>>> Karun Sharma wrote:
>>>>
>>>>
>>>>> Hi Sean:
>>>>>
>>>>> If you are using OFED 1.1 release, you will find a "README.txt"
>>>>>
> file
>
>>>>> (/usr/local/ofed/ directory). In this file, configuration steps of
>>>>> SDP are mentioned.
>>>>> 1. modprobe ib_sdp
>>>>> 2. export LD_PRELOAD=/usr/local/ofed/lib/libsdp.so
>>>>> 3. export LIBSDP_CONFIG_FILE=/usr/local/ofed/etc/libsdp.conf
>>>>> <application name>.
>>>>>
>>>>> You may need to modify libsdp.conf before step 3.
>>>>>
>>>>> Thanks
>>>>> Karun
>>>>>
>>>>>
>>>>>
>>>>>
> --------------------------------------------------------------------
>
>>>>> ----
>>>>>
>>>>> *From:* openib-general-bounces at openib.org on behalf of Sean
>>>>>
> Hubbell
>
>>>>> *Sent:* Tue 11/28/2006 10:34 PM
>>>>> *To:* openib-general at openib.org
>>>>> *Subject:* [openib-general] Configuration of sdp
>>>>>
>>>>> Hello,
>>>>>
>>>>> I have a question... we have tested our network on our cluster
>>>>> using iperf v2.0.2.
>>>>>
>>>>> Our first test was using Gig-E network and the results were 941
>>>>>
> Mbps.
>
>>>>> (i.e. iperf -c <ipaddress> and iperf -s)
>>>>>
>>>>> Our second test was using Gig-E network and the results were 1.05
>>>>> Mbps using UDP.
>>>>> (i.e. iperf -c <ipaddress> -u and iperf -s -u)
>>>>>
>>>>> Our third test was using IBv4 network and the results was a max of
>>>>> 3.8 Mbps using TCP.
>>>>> (i.e. iperf -c <ipaddress> -P[1-16] and iperf -s)
>>>>>
>>>>> Loaded ib_sdp module and then tested:
>>>>> Our fourth test was using IBv4 network and the results was a max
>>>>>
> of
>
>>>>> 4.16
>>>>> Mbps using UDP.
>>>>> (i.e. iperf -c <ipaddress> -b1000M -P[1-16] and iperf -s -u)
>>>>>
>>>>> UNLOADED ib_sdp module and then tested:
>>>>> Our fourth test was using IBv4 network and the results was a max
>>>>>
> of
>
>>>>> 4.16
>>>>> Mbps using UDP.
>>>>> (i.e. iperf -c <ipaddress> -b1000M -P[1-16] and iperf -s -u)
>>>>>
>>>>> My question is what impact would it have on our network if
>>>>> configured sdp and is there an example of how to configure sdp
>>>>> somewhere on the wiki?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Sean
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> openib-general mailing list
>>>>> openib-general at openib.org
>>>>> http://openib.org/mailman/listinfo/openib-general
>>>>>
>>>>> To unsubscribe, please visit
>>>>> http://openib.org/mailman/listinfo/openib-general
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> openib-general mailing list
>>>> openib-general at openib.org
>>>> http://openib.org/mailman/listinfo/openib-general
>>>>
>>>> To unsubscribe, please visit
>>>> http://openib.org/mailman/listinfo/openib-general
>>>>
>>>>
>>>
>
>
>
More information about the general
mailing list