<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.3243" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=921341017-18022008><FONT face=Arial
color=#0000ff size=2>Chuck, the SM in the Topspin/Cisco switch can be turned off
from CLI, Web UI, or Element Manager UI. </FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=921341017-18022008><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=921341017-18022008><FONT face=Arial
color=#0000ff size=2>There was a SM bug last year where DDR throughput was not
as high as it should be sometimes, what does "show version" display from the
Topspin/Cisco switch CLI?</FONT></SPAN></DIV>
<DIV> </DIV>
<DIV align=left><FONT face=Arial size=2><FONT size=2>
<P align=left>Scott Weitzenkamp<BR>SQA and Release Manager<BR>Server
Virtualization Business Unit<BR>Cisco Systems<BR></P></FONT></FONT></DIV>
<DIV> </DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> general-bounces@lists.openfabrics.org
[mailto:general-bounces@lists.openfabrics.org] <B>On Behalf Of </B>Chuck
Hartley<BR><B>Sent:</B> Monday, February 18, 2008 7:19 AM<BR><B>To:</B> Hal
Rosenstock<BR><B>Cc:</B> OpenFabrics General<BR><B>Subject:</B> Re:
[ofa-general] Performance of UDAPL RDMA vs IB verbs<BR></FONT><BR></DIV>
<DIV></DIV>Thanks for the tips on the commands - that was my next
question.<BR><BR>> Is the rest of the subnet pure DDR and a mix of SDR/DDR
?<BR><BR>The rest of the network is a mixture of SDR/DDR. Here is what
we have:<BR><BR>]# ibnodes -V<BR>Ca :
0x0005ad00001dc9e4 ports 2 " HCA-1"<BR>Ca :
0x0005ad00001dc970 ports 2 " HCA-1"<BR>Ca :
0x0020c28001067759 ports 1 "IB900 TMS Infiniband
interface"<BR>Ca : 0x0020c28002067759 ports 1
"IB900 TMS Infiniband interface"<BR>Ca :
0x0020c28003067759 ports 1 "IB900 TMS Infiniband
interface"<BR>Ca : 0x0020c28004067759 ports 1
"IB900 TMS Infiniband interface"<BR>Ca :
0x0030487a06600000 ports 1 "Linux62 HCA-1"<BR>Ca
: 0x0030487a226a0000 ports 1 "linux64
HCA-1"<BR>Ca : 0x0030487a071e0000 ports 1
"Linux61 HCA-1"<BR>Ca : 0x0002c9020023bb14 ports
1 "Linux60 HCA-1"<BR>Ca : 0x0030487a2a480000
ports 1 "Linux63 HCA-1"<BR>Switch : 0x000b8cffff00441c ports 24 "MT47396
Infiniscale-III Mellanox Technologies" base port 0 lid 3 lmc 0<BR>Switch
: 0x0005ad0000094076 ports 24 "Topspin Switch" enhanced port 0 lid 15 lmc
0<BR><BR>The first two nodes in the list are the SDR blades off of the Topspin
switch. The Linux boxes are all Mellanox DDR HCAs and the IB900
interfaces are SDR. The Linux and IB900 ports are all connected to the
Mallanox switch. The Topspin switch is connected to the Mellanox
switch.<BR><BR><BR>> Does it work "right" with OpenSM off/using the vendor
SM ?<BR><BR>No, it does not work with only the SM in the Topspin switch
running. That was the initial configuration we had. Then I started
OpenSM on one of the Linux boxes and that did not work any better. I
also tried disconnection the TMS IB900 so there was only DDR devices on the
subnet and that did not make a difference either. It seems like the SM
in the Topspin switch only understands SDR maybe. There does not appear
to be an option for turning off the SM in the Topspin switch, however you can
change its priority relative to other SM on the subnet. That is
what I did and now the OpenSM is the master.<BR><BR>I didn't see
anything in the opensm.conf file that indicates that OpenSM has a concept of
priority. Is there some way to force it to always be the master?
Is there some advantage or disadvantage to running multiple copies of OpenSM
on the subnet? If you have multiple switches connected as we do,
should some of the default settings on opensm.conf be changed? In particular,
should REASSIGN_LIDS be set to
"yes"?<BR><BR>Chuck<BR><BR></BLOCKQUOTE></BODY></HTML>