<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.3243" name=GENERATOR></HEAD>
<BODY>
<DIV><SPAN class=642032009-14062009><FONT face=Arial color=#0000ff
size=2>Please, send output for both machines, so we could
compare.</FONT></SPAN></DIV>
<DIV><SPAN class=642032009-14062009><FONT face=Arial color=#0000ff size=2>I'd
also suggest to configure the interfaces statically to exclude the influence of
DHCP.</FONT></SPAN></DIV><BR>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Tzachi Dar <BR><B>Sent:</B> Sunday, June
14, 2009 12:03 PM<BR><B>To:</B> Ashwath Narasimhan; Leonid
Keller<BR><B>Cc:</B> Fab Tillier; ofw@lists.openfabrics.org<BR><B>Subject:</B>
RE: [ofw] Setting up Infiniband over WinXp- Help Needed<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV><SPAN class=793135108-14062009><FONT face=Arial color=#0000ff
size=2>Hi,</FONT></SPAN></DIV>
<DIV><SPAN class=793135108-14062009><FONT face=Arial color=#0000ff
size=2></FONT></SPAN> </DIV>
<DIV><SPAN class=793135108-14062009><FONT face=Arial color=#0000ff size=2>It
seems that your problem comes from some kind of firewall/filter/antivirus that
is installed on one machine but not the others.</FONT></SPAN></DIV>
<DIV><SPAN class=793135108-14062009><FONT face=Arial color=#0000ff
size=2></FONT></SPAN> </DIV>
<DIV><SPAN class=793135108-14062009><FONT face=Arial color=#0000ff size=2>In
order for us to get more information, please do the
followings:</FONT></SPAN></DIV>
<DIV><SPAN class=793135108-14062009><FONT face=Arial color=#0000ff size=2>1)
Delete the arp tables on both machines (run "arp -d"), than start wireshark on
both and ping from machine a to b.</FONT></SPAN></DIV>
<DIV><SPAN class=793135108-14062009><FONT face=Arial color=#0000ff
size=2>2) <SPAN class=793135108-14062009><FONT face=Arial color=#0000ff
size=2>Delete the arp tables on both machines (run "arp -d"), than start
wireshark on both and ping from machine b to
a.</FONT></SPAN></FONT></SPAN></DIV>
<DIV><SPAN class=793135108-14062009><FONT face=Arial color=#0000ff
size=2><SPAN class=793135108-14062009>Please send me the captures of these two
experiments. (you should have 4 files).</SPAN></FONT></SPAN></DIV>
<DIV><SPAN class=793135108-14062009><FONT face=Arial color=#0000ff
size=2><SPAN class=793135108-14062009></SPAN></FONT></SPAN> </DIV>
<DIV><SPAN class=793135108-14062009><SPAN
class=793135108-14062009></SPAN></SPAN><FONT face=Arial><FONT
color=#0000ff><FONT size=2>C<SPAN class=793135108-14062009>an you try using a
3rd computer and see how it works?</SPAN></FONT></FONT></FONT></DIV>
<DIV><FONT face=Arial><FONT color=#0000ff><FONT size=2><SPAN
class=793135108-14062009></SPAN></FONT></FONT></FONT> </DIV>
<DIV><FONT face=Arial><FONT color=#0000ff><FONT size=2><SPAN
class=793135108-14062009>Thanks</SPAN></FONT></FONT></FONT></DIV>
<DIV><FONT face=Arial><FONT color=#0000ff><FONT size=2><SPAN
class=793135108-14062009>Tzachi</SPAN></FONT></FONT></FONT></DIV>
<DIV><BR></DIV>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> nashwath@gmail.com
[mailto:nashwath@gmail.com] <B>On Behalf Of </B>Ashwath
Narasimhan<BR><B>Sent:</B> Friday, June 12, 2009 4:15 AM<BR><B>To:</B>
Leonid Keller<BR><B>Cc:</B> Tzachi Dar; Fab Tillier;
ofw@lists.openfabrics.org<BR><B>Subject:</B> Re: [ofw] Setting up Infiniband
over WinXp- Help Needed<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV>Hi Everyone,</DIV>
<DIV> </DIV>
<DIV> Thank you so much for your replies. Still the same problem.. able
to ping sucessfully from one side but not from the other.</DIV>
<DIV> </DIV>
<DIV>Hi Tzachi and Leonid,</DIV>
<DIV>a. I followed your steps. I am able to view the infiniband data when I
run the server (ib_send_bw -a) on computer 2 and I connect to this from
computer 1 (ib_send_bw -a <ip>). However, I do not view this data when
I run server on computer 1 and connect from computer 2. I get a
pp_connect_sock<ip,port> failed in the latter case.</DIV>
<DIV> </DIV>
<DIV>b. I disabled and enabled network interfaces on both ports, but no
luck. It still doesnt work.</DIV>
<DIV> </DIV>
<DIV>c. I know that its <STRONG>not</STRONG> a hardware issue because the
same problem persists when I interchange the infiniband cards i.e. the card
that was actually plugged into computer 2 is now plugged into computer 1 and
vice versa. I get the same issue in this case too. </DIV>
<DIV> </DIV>
<DIV>d. I then installed Ultra VNC and ran one end as server and the other
as client.. <STRONG>And it worked perfectly fine!!!!!!!</STRONG>.. Both from
computer 1 to computer 2 and computer 2 to computer 1. I then installed
WIRESHARK on both computers. I could see the Computer 2 send the Ping
requests to computer 1 in Computer 2's Wireshark window but for some
bizzare reason computer 1 was rejecting these ping requests. When I checked
the connection status of Computer 1, I could see the number of received
packets also increasing but Computer 1 did not send back any packets.
</DIV>
<DIV> </DIV>
<DIV>e. I suspect this issue is arising because of some win xp setting in
Computer 1. There is no difference between the two PC's. both are brand new
PC's having Xp. The only difference is that I have a wifi driver on computer
1. all my firewall settings are disabled. I even uninstalled my wifi driver,
but still the same problem persists. </DIV>
<DIV> </DIV>
<DIV>f. ipconfig and vstat return the correct values.. here's the output of
these commands on computer 1</DIV>
<DIV><BR>Windows IP Configuration</DIV>
<DIV> Host Name . . . . . . . . .
. . . : LENOVO-CF61BEED<BR>
Primary Dns Suffix . . . . . . .
:<BR> Node Type . . . . . . . . .
. . . : Unknown<BR> IP Routing
Enabled. . . . . . . . : No<BR>
WINS Proxy Enabled. . . . . . . . :
No<BR> DNS Suffix Search List. . .
. . . : <A href="http://ee.columbia.edu">ee.columbia.edu</A></DIV>
<DIV>Ethernet adapter Local Area Connection:</DIV>
<DIV> Connection-specific DNS
Suffix . : <A
href="http://ee.columbia.edu">ee.columbia.edu</A><BR>
Description . . . . . . . . . . . : Marvell Yukon 88E8056 PCI-E
Gigabit<BR>Ethernet Controller<BR>
Physical Address. . . . . . . . . :
00-21-97-CB-64-97<BR> Dhcp
Enabled. . . . . . . . . . . :
Yes<BR> Autoconfiguration Enabled
. . . . : Yes<BR> IP Address. . .
. . . . . . . . . :
128.59.65.132<BR> Subnet Mask . .
. . . . . . . . . :
255.255.252.0<BR> Default Gateway
. . . . . . . . . :
128.59.64.1<BR> DHCP Server . . .
. . . . . . . . : 128.59.64.59<BR>
DNS Servers . . . . . . . . . . . :
128.59.64.59<BR>
128.59.16.20<BR> Lease Obtained. .
. . . . . . . . : Thursday, June 11, 2009 5:39:25
PM<BR> Lease Expires . . . . . . .
. . . : Saturday, June 13, 2009 12:39:25 PM</DIV>
<DIV>Ethernet adapter Local Area Connection 7:</DIV>
<DIV> Media State . . . . . . . .
. . . : Media disconnected<BR>
Description . . . . . . . . . . . : Mellanox IPoIB Adapter
#4<BR> Physical Address. . . . . .
. . . : 00-05-AD-04-E7-C6</DIV>
<DIV>Ethernet adapter Local Area Connection 6:</DIV>
<DIV> Connection-specific DNS
Suffix . :<BR> Description .
. . . . . . . . . . : Mellanox IPoIB Adapter
#3<BR> Physical Address. . . . . .
. . . : 00-05-AD-04-E7-C5<BR> Dhcp
Enabled. . . . . . . . . . . :
Yes<BR> Autoconfiguration Enabled
. . . . : Yes<BR>
Autoconfiguration IP Address. . . :
169.254.53.191<BR> Subnet Mask . .
. . . . . . . . . : 255.255.0.0 </DIV>
<DIV> Default Gateway . . . .
. . . . . :<BR><BR>C:\<my directory>vstat</DIV>
<DIV>
hca_idx=0<BR> uplink={BUS=PCI_E,
SPEED=2.5 Gbps,<BR>
vendor_id=0x05ad<BR>
vendor_part_id=0x6278<BR>
hw_ver=0xa0<BR>
fw_ver=0x400080395<BR>
node_guid=0005:ad00:0004:e7c4<BR>
num_phys_ports=2<BR>
port=1<BR>
port_state=PORT_ACTIVE
(4)<BR>
link_speed=2.5 Gbps
(1)<BR>
link_width=4x
(2)<BR>
rate=10
Gbps<BR>
port_phys_state=LINK_UP
(5)<BR>
active_speed=2.5 Gbps
(1)<BR>
sm_lid=0x0001<BR>
port_lid=0x0002<BR>
port_lmc=0x0<BR>
max_mtu=2048 (4)</DIV>
<DIV>
port=2<BR>
port_state=PORT_DOWN
(1)<BR>
link_speed=NA<BR>
link_width=NA<BR>
rate=NA<BR>
port_phys_state=POLLING
(2)<BR>
active_speed=2.5 Gbps
(1)<BR>
sm_lid=0x0000<BR>
port_lid=0x0000<BR>
port_lmc=0x0<BR>
max_mtu=2048 (4)</DIV>
<DIV> </DIV>
<DIV>P.S. I am using the first port.</DIV>
<DIV> </DIV>
<DIV>regards,</DIV>
<DIV>Ashwath</DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV><BR> </DIV>
<DIV class=gmail_quote>On Thu, Jun 11, 2009 at 7:00 AM, Leonid Keller <SPAN
dir=ltr><<A href="mailto:leonid@mellanox.co.il"
target=_blank>leonid@mellanox.co.il</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV>
<DIV><SPAN><FONT face=Arial color=#0000ff size=2>Hi
Ashwath,</FONT></SPAN></DIV>
<DIV><SPAN><FONT face=Arial color=#0000ff
size=2></FONT></SPAN> </DIV>
<DIV><SPAN><FONT face=Arial color=#0000ff size=2>If you still have
problems, send us, please, the output of 'vstat -v' and 'ipconfig /all' on
both machines.</FONT></SPAN></DIV>
<DIV><SPAN><FONT face=Arial color=#0000ff
size=2></FONT></SPAN> </DIV>
<DIV><SPAN><FONT face=Arial color=#0000ff size=2>TIA</FONT></SPAN></DIV>
<DIV><SPAN><FONT face=Arial color=#0000ff
size=2>Leonid</FONT></SPAN></DIV><BR>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV lang=en-us dir=ltr align=left>
<HR>
<FONT face=Tahoma size=2><B>From:</B> <A
href="mailto:ofw-bounces@lists.openfabrics.org"
target=_blank>ofw-bounces@lists.openfabrics.org</A> [mailto:<A
href="mailto:ofw-bounces@lists.openfabrics.org"
target=_blank>ofw-bounces@lists.openfabrics.org</A>] <B>On Behalf Of
</B>Tzachi Dar<BR><B>Sent:</B> Thursday, June 11, 2009 4:43
PM<BR><B>To:</B> Ashwath Narasimhan; Fab Tillier
<DIV><BR><B>Cc:</B> <A href="mailto:ofw@lists.openfabrics.org"
target=_blank>ofw@lists.openfabrics.org</A><BR></DIV><B>Subject:</B> RE:
[ofw] Setting up Infiniband over WinXp- Help Needed<BR></FONT><BR></DIV>
<DIV>
<DIV></DIV>
<DIV>
<DIV></DIV>
<DIV><SPAN><SPAN lang=EN>
<P><FONT face=Arial color=#0000ff size=2>Hi Ashwath,</FONT></P>
<P><SPAN><FONT face=Arial color=#0000ff size=2>There are a few things
that I would like you to try:</FONT></SPAN></P>
<P><SPAN><FONT face=Arial color=#0000ff size=2>1) Please run some low
level IB test to see that traffic is indeed ok. On one computer please
run </FONT></SPAN></P>
<P><SPAN><FONT face=Arial color=#0000ff size=2>ib_send_bw
-a</FONT></SPAN></P>
<P><SPAN><FONT face=Arial color=#0000ff size=2>and on the other computer
please run</FONT></SPAN></P></SPAN></SPAN></DIV>
<DIV><FONT face=Arial size=2><SPAN><SPAN><FONT face=Arial size=2><FONT
color=#0000ff>ib_send_bw -a <FONT face="Times New Roman"
size=3>192.168.0.x
(where x is the ip of the remote side. Please start this test with
the <STRONG>Ethernet</STRONG> addresses of the
ports).</FONT></FONT></FONT></SPAN></SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff
size=2><SPAN><SPAN></SPAN></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN><SPAN>2) Assuming all
works well please try to disable and enable the network interfaces
(ipoib) on both ports. Please see if this
helps.</SPAN></SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff
size=2><SPAN></SPAN></FONT> </DIV>
<DIV><FONT color=#0000ff><FONT face=Arial size=2><SPAN>3) If this
doesn't help, you will probably need to change the parameter of
"</SPAN></FONT><FONT face=Arial size=2><SPAN>Guid bitwise mask" to e7.
To do this, please open the device manager, than go to "network
adapters" select the ipoib interfaces and than right click properties.
Select the "<FONT face=Arial size=2><SPAN>Guid bitwise mask" and change
it to e7.</SPAN></FONT></SPAN></FONT></FONT></DIV>
<DIV><FONT face=Arial size=2><SPAN><FONT face=Arial color=#0000ff
size=2><SPAN></SPAN></FONT></SPAN></FONT> </DIV>
<DIV><FONT face=Arial size=2><SPAN><FONT face=Arial color=#0000ff
size=2><SPAN>If all doesn't help, can you give me remote access to these
stations?</SPAN></FONT></SPAN></FONT></DIV>
<DIV><FONT face=Arial size=2><SPAN><FONT face=Arial color=#0000ff
size=2><SPAN></SPAN></FONT></SPAN></FONT> </DIV>
<DIV><FONT face=Arial size=2><SPAN><FONT face=Arial color=#0000ff
size=2><SPAN>Thanks</SPAN></FONT></SPAN></FONT></DIV>
<DIV><FONT face=Arial size=2><SPAN><FONT face=Arial color=#0000ff
size=2><SPAN>Tzachi</SPAN></FONT></SPAN></FONT></DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV lang=en-us dir=ltr align=left>
<HR>
<FONT face=Tahoma size=2><B>From:</B> <A
href="mailto:ofw-bounces@lists.openfabrics.org"
target=_blank>ofw-bounces@lists.openfabrics.org</A> [mailto:<A
href="mailto:ofw-bounces@lists.openfabrics.org"
target=_blank>ofw-bounces@lists.openfabrics.org</A>] <B>On Behalf Of
</B>Ashwath Narasimhan<BR><B>Sent:</B> Thursday, June 11, 2009 5:04
AM<BR><B>To:</B> Fab Tillier<BR><B>Cc:</B> <A
href="mailto:ofw@lists.openfabrics.org"
target=_blank>ofw@lists.openfabrics.org</A><BR><B>Subject:</B> Re:
[ofw] Setting up Infiniband over WinXp- Help
Needed<BR></FONT><BR></DIV>
<DIV></DIV>Hi
Fab,<BR><BR>
I restarted opensm on the other node. I ran both opensm and ibdiagnet
on the other node (not on the node where opensm is running). The logs
are similar to the one I attached in my previous mail. (Computer 1
:-192.168.0.1 logs in my previous mail). I have disabled firewall
settings on both nodes. However, I still cannot get it to work.
I cannot access the shared folder of each node from the other.
Is there something else I can try?<BR><BR>p.s. There is a typo in my
previous mail. I had opensm running on computer 2 and not computer
1.<BR><BR>regards,<BR>Ashwath<BR><BR>
<DIV class=gmail_quote>On Wed, Jun 10, 2009 at 9:42 PM, Fab Tillier
<SPAN dir=ltr><<A href="mailto:ftillier@windows.microsoft.com"
target=_blank>ftillier@windows.microsoft.com</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">Hi
Ashwath,<BR>
<DIV><BR>>I am new to the world of infiniband and I am trying to
set up an<BR>>infiniband network between two Lenovo x86 Desktops
(Windows Xp).<BR><BR></DIV>Welcome!<BR>
<DIV><BR>>Problem:-<BR>>I am able to ping 192.168.0.2 from
192.168.0.1 however ping does not<BR>>work the other way around
i.e. from 192.168.0.2 to 192.168.0.1. I don't<BR>>understand why
this is not happening. I see that the "bind" fails but I<BR>>dont
understand why. Shouldn't it be two way? (I am using one cable
to<BR>>connect the two adaptors) Please help me.
Thanks.<BR><BR></DIV>Check your firewall settings on the 192.168.0.1
box. Can you access the administrative share on each node from
the other (\\192.168.0.1\c$, and \\192.168.0.2\c$?)<BR>
<DIV><BR>>========================================================================<BR>>Computer
No2: 192.168.0.2<BR>>when I ran osmtest here:<BR>><BR>>
C:\<mydirectory>osmtest -f -a<BR>>Command Line
Arguments<BR>>Done with args<BR>>
Flow = All Validations<BR>>Using default guid
0x5ad000004e7c6<BR>>[17:59:17:437][0388] -> osm_vendor_bind:
Binding to port<BR>>0x5ad000004e7c6.<BR>>[17:59:17:437][0388]
-> osm_vendor_bind: ERR 3B21: Unable to register<BR>>QP0 MAD
se<BR>>rvice
(IB_INSUFFICIENT_MEMORY).<BR>>[17:59:17:437][0388] ->
osmv_bind_sa: ERR 0506: Fail to bind to
vendor<BR>>SMI.<BR>>[17:59:17:437][0388] -> osmtest_bind:
ERR 0137: Unable to bind to SA<BR><BR></DIV>You probably have OpenSM
running on this node, yes? You can't run osmtest on the same
port where OpenSM is running.<BR>
<DIV><BR>>when I ran ibdiagnet here<BR>><BR>>C:\<my
directory>ibdiagnet<BR>>Loading IBIS from:
C:/Program<BR>>Files/Mellanox/MLNX_WinOF/Tools/ibdiagnet.exe/lib/<BR>>ibis1.0<BR>>Loading
IBDM from:
C:/Program<BR>>Files/Mellanox/MLNX_WinOF/Tools/ibdiagnet.exe/lib/<BR>>ibdm1.0<BR>>-W-
Topology file is not specified.<BR>> Reports
regarding cluster links will use direct routes.<BR>>-I- Using
port 2 as the local port.<BR>>-E- Fail to
ibsac_bind.<BR><BR></DIV>Don't know the details of this tool, maybe
it's running into the same problems as osmtest? Try running
OpenSM on the other node and see if the problem follows the SM or
not.<BR><FONT
color=#888888><BR>-Fab<BR></FONT></BLOCKQUOTE></DIV><BR><BR
clear=all><BR>--
<BR>regards,<BR>Ashwath<BR></BLOCKQUOTE></DIV></DIV></BLOCKQUOTE></DIV></BLOCKQUOTE></DIV><BR><BR
clear=all>
<DIV></DIV><BR>--
<BR>regards,<BR>Ashwath<BR></BLOCKQUOTE></BLOCKQUOTE></BODY></HTML>