<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=US-ASCII">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2654.45">
<TITLE>RE: [openib-general] OpenSM (again)</TITLE>
</HEAD>
<BODY>
<P><FONT SIZE=2>Hi Roland,</FONT>
</P>
<P><FONT SIZE=2>If the case is reproducible, please run "opensm -V" and send us the osm.log</FONT>
</P>
<P><FONT SIZE=2>Thanks</FONT>
</P>
<P><FONT SIZE=2>Eitan Zahavi</FONT>
</P>
<P><FONT SIZE=2>> -----Original Message-----</FONT>
<BR><FONT SIZE=2>> From: Roland Fehrenbacher [<A HREF="mailto:rf@q-leap.de">mailto:rf@q-leap.de</A>]</FONT>
<BR><FONT SIZE=2>> Sent: Monday, April 11, 2005 7:28 PM</FONT>
<BR><FONT SIZE=2>> To: openib-general@openib.org</FONT>
<BR><FONT SIZE=2>> Subject: [openib-general] OpenSM (again)</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Hi,</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> I got gen2 opensm running fine now (there was a problem with a wrong</FONT>
<BR><FONT SIZE=2>> include file), and managed to get IP running on a network of</FONT>
<BR><FONT SIZE=2>> currently 40 machines (final size will be 144). Performance is pretty</FONT>
<BR><FONT SIZE=2>> impressive (initial tests with a simple netpipe): I got a latency of</FONT>
<BR><FONT SIZE=2>> 18microsec, and a maximum throughput of approx. 400MB/sec at packet</FONT>
<BR><FONT SIZE=2>> size approx. 1MB which then levels of at about 340MB/s for larger</FONT>
<BR><FONT SIZE=2>> packets.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> One problem and two questions:</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Problem: When I reboot all the 40 nodes (apart from the one the opensm</FONT>
<BR><FONT SIZE=2>> is running), the network is non-functional (no pings go through, even</FONT>
<BR><FONT SIZE=2>> though ports show status "Active") for quite a while (more than 10</FONT>
<BR><FONT SIZE=2>> minutes) after all the nodes have come up. It then recovers without</FONT>
<BR><FONT SIZE=2>> intervention. Is this normal? Single node reboots don't affect the</FONT>
<BR><FONT SIZE=2>> network operation. osm Log file is appended.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Question 1: Can I run opensm in a master slave configuration? I noticed</FONT>
<BR><FONT SIZE=2>> that there is a priority commandline option, but am not sure how to</FONT>
<BR><FONT SIZE=2>> apply this.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Question 2: I plan to run the gen1/Mellanox IBGD drivers on the</FONT>
<BR><FONT SIZE=2>> compute nodes (need fast MPI), and gen2 on the control/storage nodes</FONT>
<BR><FONT SIZE=2>> (need only IP) with gen2 opensm running on the control nodes. Is there</FONT>
<BR><FONT SIZE=2>> any reason why this should not work reliably?</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Roland</FONT>
</P>
</BODY>
</HTML>