<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:x =
"urn:schemas-microsoft-com:office:excel" xmlns:w =
"urn:schemas-microsoft-com:office:word" xmlns:o =
"urn:schemas-microsoft-com:office:office" xmlns:v =
"urn:schemas-microsoft-com:vml"><HEAD><!--[if !mso]>
<META content="text/html; charset=us-ascii" http-equiv=Content-Type>
<STYLE>v\:* {
BEHAVIOR: url(#default#VML)
}
o\:* {
BEHAVIOR: url(#default#VML)
}
w\:* {
BEHAVIOR: url(#default#VML)
}
.shape {
BEHAVIOR: url(#default#VML)
}
</STYLE>
<![endif]-->
<STYLE>@font-face {
font-family: Tahoma;
}
@page Section1 {size: 8.5in 11.0in; margin: 1.0in 1.25in 1.0in 1.25in; }
P.MsoNormal {
MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"; FONT-SIZE: 12pt
}
LI.MsoNormal {
MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"; FONT-SIZE: 12pt
}
DIV.MsoNormal {
MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"; FONT-SIZE: 12pt
}
A:link {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline
}
SPAN.EmailStyle17 {
FONT-FAMILY: Arial; COLOR: windowtext; mso-style-type: personal
}
SPAN.EmailStyle18 {
FONT-FAMILY: Arial; COLOR: navy; mso-style-type: personal-reply
}
DIV.Section1 {
page: Section1
}
</STYLE>
<META name=GENERATOR content="MSHTML 8.00.6001.18702"></HEAD>
<BODY lang=EN-US link=blue vLink=purple>
<DIV><SPAN class=615072515-12112009><FONT color=#0000ff size=2 face=Arial>As for
the ipoib connecting to the broadcast group issue. Can another instances of
ipoib connect to that broadcast group ? (for example machines that have the ndis
5.1 version installed?).</FONT></SPAN></DIV>
<DIV><SPAN class=615072515-12112009><FONT color=#0000ff size=2
face=Arial></FONT></SPAN> </DIV>
<DIV><SPAN class=615072515-12112009><FONT color=#0000ff size=2 face=Arial>As for
the dump file. We are not able to locate the correct symbols. What version have
you been using? We get the following error: </FONT></SPAN></DIV>
<DIV><SPAN class=615072515-12112009> <FONT color=#0000ff
size=2 face=Arial><A
href="http://www.openfabrics.org/downloads/WinOF/v2.1/Server_2008-Vista-HPC/SymStor//mlx4_bus.sys/4A16ADB132000/mlx4_bus.sys">http://www.openfabrics.org/downloads/WinOF/v2.1/Server_2008-Vista-HPC/SymStor//mlx4_bus.sys/4A16ADB132000/mlx4_bus.sys</A>
not found<BR>So we can not get the true symbols. Can you please run !analyze -v
and send us the results.</FONT></SPAN></DIV>
<DIV><SPAN class=615072515-12112009><FONT color=#0000ff size=2
face=Arial></FONT></SPAN> </DIV>
<DIV><SPAN class=615072515-12112009><FONT color=#0000ff size=2
face=Arial> </DIV></FONT></SPAN>
<DIV><SPAN class=615072515-12112009><FONT color=#0000ff size=2 face=Arial>As for
the system not booting. Can you load it using WinPE and copy the <FONT
color=#000080>mlx4_hca.sys file to it. I hope that after that you will be able
to load.</FONT></FONT></SPAN></DIV>
<DIV><SPAN class=615072515-12112009><FONT color=#0000ff size=2 face=Arial><FONT
color=#000080></FONT></FONT></SPAN> </DIV>
<DIV><SPAN class=615072515-12112009><FONT color=#0000ff size=2 face=Arial><FONT
color=#000080>Thanks</FONT></FONT></SPAN></DIV>
<DIV><SPAN class=615072515-12112009><FONT color=#0000ff size=2 face=Arial><FONT
color=#000080>Tzachi</FONT></DIV></FONT></SPAN><BR>
<BLOCKQUOTE
style="BORDER-LEFT: #0000ff 2px solid; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px"
dir=ltr>
<DIV dir=ltr lang=en-us class=OutlookMessageHeader align=left>
<HR tabIndex=-1>
<FONT size=2 face=Tahoma><B>From:</B> Anatoly Greenblatt
[mailto:anatolyg@voltaire.com] <BR><B>Sent:</B> Thursday, November 12, 2009
1:44 PM<BR><B>To:</B> Tzachi Dar; Smith, Stan;
ofw@lists.openfabrics.org<BR><B>Subject:</B> RE: [ofw] RE:
installation/connectivity problems on hpc server<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV class=Section1>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt">Hi,<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt">We had 2 (and now
even more) problems on these systems.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt">1<SUP>st</SUP> is the
problem of installation but we have a workaround.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt">2<SUP>nd</SUP> is the
ipoib problem system log shows: “</SPAN></FONT><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">Mellanox IPoIB Adapter #3: Subnet
Administrator failed query for broadcast group information.<FONT
color=navy><SPAN
style="COLOR: navy">”</SPAN></FONT><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt">3<SUP>rd</SUP> two
systems had bsod (minidump is attached). After bsod, one system stops booting
and the text screen shows that “system failed to boot because critical system
driver is missing: mlx4_hca.sys<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt">Regardsm<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt">Anatoly.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: navy; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<DIV>
<DIV style="TEXT-ALIGN: center" class=MsoNormal align=center><FONT size=3
face="Times New Roman"><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center SIZE=2 width="100%">
</SPAN></FONT></DIV>
<P class=MsoNormal><B><FONT size=2 face=Tahoma><SPAN
style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt; FONT-WEIGHT: bold">From:</SPAN></FONT></B><FONT
size=2 face=Tahoma><SPAN style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt"> Tzachi
Dar [mailto:tzachid@mellanox.co.il] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Thursday, November 12, 2009 12:19
PM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Smith, Stan; Anatoly
Greenblatt; ofw@lists.openfabrics.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE: [ofw] RE:
installation/connectivity problems on hpc
server</SPAN></FONT><o:p></o:p></P></DIV>
<P class=MsoNormal><FONT size=3 face="Times New Roman"><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<DIV>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt">IMHO, if vstat shows
the links as up, this is not an installation problem but rather an ipoib
problem.</SPAN></FONT><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><FONT size=3 face="Times New Roman"><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<DIV>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt">Can you please run
ipoib with trace and send us the logs (also, do you have anything in the event
viewer)?</SPAN></FONT><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><FONT size=3 face="Times New Roman"><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<DIV>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt">Thanks</SPAN></FONT><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt">Tzachi</SPAN></FONT><o:p></o:p></P></DIV>
<BLOCKQUOTE
style="BORDER-BOTTOM: medium none; BORDER-LEFT: blue 1.5pt solid; PADDING-BOTTOM: 0in; MARGIN: 5pt 0in 5pt 3.75pt; PADDING-LEFT: 4pt; PADDING-RIGHT: 0in; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0in">
<P class=MsoNormal><FONT size=3 face="Times New Roman"><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<DIV style="TEXT-ALIGN: center" class=MsoNormal align=center><FONT size=3
face="Times New Roman"><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center SIZE=2 width="100%">
</SPAN></FONT></DIV>
<P style="MARGIN-BOTTOM: 12pt" class=MsoNormal><B><FONT size=2
face=Tahoma><SPAN
style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt; FONT-WEIGHT: bold">From:</SPAN></FONT></B><FONT
size=2 face=Tahoma><SPAN style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt">
ofw-bounces@lists.openfabrics.org [mailto:ofw-bounces@lists.openfabrics.org]
<B><SPAN style="FONT-WEIGHT: bold">On Behalf Of </SPAN></B>Smith,
Stan<BR><B><SPAN style="FONT-WEIGHT: bold">Sent:</SPAN></B> Thursday,
November 12, 2009 12:35 AM<BR><B><SPAN
style="FONT-WEIGHT: bold">To:</SPAN></B> Anatoly Greenblatt;
ofw@lists.openfabrics.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Subject:</SPAN></B> [ofw] RE:
installation/connectivity problems on hpc
server</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt">Hello,</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt"> Please see
inline comments.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT size=3 face="Times New Roman"><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<DIV style="TEXT-ALIGN: center" class=MsoNormal align=center><FONT size=3
face="Times New Roman"><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center SIZE=2 width="100%">
</SPAN></FONT></DIV>
<P style="MARGIN-BOTTOM: 12pt" class=MsoNormal><B><FONT size=2
face=Tahoma><SPAN
style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt; FONT-WEIGHT: bold">From:</SPAN></FONT></B><FONT
size=2 face=Tahoma><SPAN style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt">
ofw-bounces@lists.openfabrics.org [mailto:ofw-bounces@lists.openfabrics.org]
<B><SPAN style="FONT-WEIGHT: bold">On Behalf Of </SPAN></B>Anatoly
Greenblatt<BR><B><SPAN style="FONT-WEIGHT: bold">Sent:</SPAN></B> Wednesday,
November 11, 2009 1:05 PM<BR><B><SPAN
style="FONT-WEIGHT: bold">To:</SPAN></B>
ofw@lists.openfabrics.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Subject:</SPAN></B> [ofw]
installation/connectivity problems on hpc
server</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">Hi,<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">Should we have any problems
installing Winof 2.1 on server 2008 hpc edition sp2? Or am I missing
something.<FONT color=blue><SPAN
style="COLOR: blue"> </SPAN></FONT></SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT size=3 face="Times New Roman"><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt">Have not attempted
a WinOF install on Svr 2008 HPC sp2, although I would not anticipate
any problems.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt">Try installing via
'start/wait msiexec /I WinOF_2-1_wlh_x64.msi /Lv msi.log'. Grep log file for
error.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">The installation ends
prematurely claiming that previous installation was detected.<FONT
color=blue><SPAN
style="COLOR: blue"> </SPAN></FONT></SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt">Is this a MSFT
installer error message or a WinOF installer error
message?</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT size=3 face="Times New Roman"><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">I’ve exctracted the drivers from
winof hpc x64 msi and installed manually.<FONT color=blue><SPAN
style="COLOR: blue"> </SPAN></FONT></SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT size=3 face="Times New Roman"><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt">Since you have
already installed HCA drivers by hand, you want to install WinOF with NO
devices installed </SPAN></FONT><FONT color=blue><SPAN
style="COLOR: blue"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt"> 'start/wait
msiexec /I WinOF_2-1_wlh_x64.msi NODEV=1' # just take the
default install, no devices will be installed.</SPAN></FONT><FONT
color=blue><SPAN style="COLOR: blue"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">The bus/hca/ipoib drivers were
installed successfully, however the ipoib network adapter shows status
“disconnected”<FONT color=blue><SPAN
style="COLOR: blue"> </SPAN></FONT></SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT color=blue size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; COLOR: blue; FONT-SIZE: 10pt">State of cable
disconnected indicates the SM has not seen/configured --> Active port
state. <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">Opsnsm is running on linux node
and shows all ports/nodes as connected. Vstat on the nodes shows that ports
that are physically connected are up.<FONT color=blue><SPAN
style="COLOR: blue"> UP == Active port
state? <o:p></o:p></SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">It is c-class hp blade with
connectx gen2. firmware 2.6.1.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">Any ideas how to fix ipoib
connection?<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">Thanks,<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt">Anatoly.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial><SPAN
style="FONT-FAMILY: Arial; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></FONT></P></BLOCKQUOTE></DIV></BLOCKQUOTE></BODY></HTML>