<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: arial,helvetica,sans-serif; font-size: 10pt; color: #000000'><br><br><hr id="zwchr"><div style="color: rgb(0, 0, 0); font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><b>From: </b>"John Valdes" <valdes@anl.gov><br><b>To: </b>"Andrei Mikhailovsky" <andrei@arhont.com><br><b>Cc: </b>users@lists.openfabrics.org<br><b>Sent: </b>Saturday, 25 May, 2013 2:19:40 AM<br><b>Subject: </b>Re: [Users] HP BLc QLogic 4X QDR IB Switch oddness<br><br>Andrei Mikhailovsky wrote:<br>> John Valdes wrote:<br>> > What's the physical topology of the IB network between the blade <br>> > servers and the switch?<br>> <br>> AM: I am not really sure. The servers do have the IB mezzanine card and from what i've read it is a pci-e card. I am unsure how the blade servers are connected to the switch. I guess it's an internal HP/QLogic interconnect. <br><br>Found some docs on Intel's website at:<br>http://www.intel.com/p/en_US/support/highlights/network/ts-fbs12100 <br>From that, it looks like the topology is very simple; the switch<br>installs in a slot in the blade chassis, and it looks like it has 16<br>internal (through the backplane of the chassis) IB connections, one<br>to each blade server in the chassis, plus 16 external QSFP ports.<br><br><br>AM: yeah, that pretty much sums up the switch<br><br><br><br>I was thinking maybe there was something odd in the topology that<br>was causing the subnet manager to fail to negotiate link properly w/<br>the blade servers. It doesn't sound like that's the case.<br><br><br>AM: I do not see any logs on the SM side when the port state changes. The only logs I see is when the port becomes Active, I see that in the logs and no errors before or prior to that<br><br><br>Does "perfquery" report errors on the blade servers' HCA port? How<br>about on the (internal) ports on the switch that connect to the blade<br>servers? To check, give perfquery the lid of the device you want to<br>check (either the server's lid or the switch's lid) and the port<br>number to check (that should always be 1 for the servers, and the<br>corresponding switch port number for the switch, 1 through 16 or 17<br>through 32 depending on which are the internal ports and which are the<br>external. "ibnetdiscover" should tell you what servers are connected<br>to what ports on the switch.<br><br><br>AM: There are occasional errors in perfqery when I go through the hca and switch ports, but from what I've read it is not a big deal if there are not that many of them.<br><br><br>John<br></div><br></div></body></html>