[Users] HP BLc QLogic 4X QDR IB Switch oddness

[Andrei Mikhailovsky]

----- Original Message -----

From: "John Valdes" <valdes at anl.gov> 
To: "Andrei Mikhailovsky" <andrei at arhont.com> 
Cc: users at lists.openfabrics.org 
Sent: Saturday, 25 May, 2013 2:19:40 AM 
Subject: Re: [Users] HP BLc QLogic 4X QDR IB Switch oddness 

Andrei Mikhailovsky wrote: 
> John Valdes wrote: 
> > What's the physical topology of the IB network between the blade 
> > servers and the switch? 
> 
> AM: I am not really sure. The servers do have the IB mezzanine card and from what i've read it is a pci-e card. I am unsure how the blade servers are connected to the switch. I guess it's an internal HP/QLogic interconnect. 

Found some docs on Intel's website at: 
http://www.intel.com/p/en_US/support/highlights/network/ts-fbs12100 
>From that, it looks like the topology is very simple; the switch 
installs in a slot in the blade chassis, and it looks like it has 16 
internal (through the backplane of the chassis) IB connections, one 
to each blade server in the chassis, plus 16 external QSFP ports. 


AM: yeah, that pretty much sums up the switch 



I was thinking maybe there was something odd in the topology that 
was causing the subnet manager to fail to negotiate link properly w/ 
the blade servers. It doesn't sound like that's the case. 


AM: I do not see any logs on the SM side when the port state changes. The only logs I see is when the port becomes Active, I see that in the logs and no errors before or prior to that 


Does "perfquery" report errors on the blade servers' HCA port? How 
about on the (internal) ports on the switch that connect to the blade 
servers? To check, give perfquery the lid of the device you want to 
check (either the server's lid or the switch's lid) and the port 
number to check (that should always be 1 for the servers, and the 
corresponding switch port number for the switch, 1 through 16 or 17 
through 32 depending on which are the internal ports and which are the 
external. "ibnetdiscover" should tell you what servers are connected 
to what ports on the switch. 


AM: There are occasional errors in perfqery when I go through the hca and switch ports, but from what I've read it is not a big deal if there are not that many of them. 


John 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20130526/b5db231d/attachment.html>