[Users] connect Mellanox IB FDR adapter to QLogic IB QDR switch?

[Andrei Mikhailovsky]

Marko, 

I am experiencing similar issues, but with HP (QLogic) blade switch and HP (Qlogic) mezzanine hca cards. The maximum time I had to wait for the link to become Active is about 20 hours, but on average it's about 15 hours to get the link. I've got guys at Intel and HP trying to help the issue, but so far there has not been any solution. 

Please let me know if you manage to find the cause of the problem. Perhaps my problem is related? 

thanks 

Andrei 
----- Original Message -----

From: "Marko Kobal" <marko.kobal at arctur.si> 
To: users at lists.openfabrics.org 
Sent: Thursday, 13 June, 2013 2:46:09 PM 
Subject: Re: [Users] connect Mellanox IB FDR adapter to QLogic IB QDR switch? 

Hi, 

Well, I noticed that the links did eventually came up at the second node 
as well ... however it took several hours to do the link initialization: 

cat /var/log/messages | grep mlx4_core 

node_1: 
Jun 6 17:42:08 node_1 kernel: mlx4_core 0000:10:00.0: PCI INT A -> GSI 
30 (level, low) -> IRQ 30 
Jun 7 04:35:33 node_1 kernel: mlx4_core 0000:10:00.0: mlx4_ib: Port 1 
logical link is up 

node_2: 
Jun 12 16:05:12 node_2 kernel: mlx4_core 0000:10:00.0: PCI INT A -> GSI 
30 (level, low) -> IRQ 30 
Jun 12 20:50:24 node_2 kernel: mlx4_core 0000:10:00.0: mlx4_ib: Port 1 
logical link is up 

The connection is now actually working, however the extremely long time 
to make the link initialization is really weird (and annoying when doing 
server or openibd restart) ... Anyone seen this kind of behaviour on 
Mellanox adapters? 


Kind regards, Marko Kobal 


On 12. 06. 2013 17:12, Marko Kobal wrote: 
> Hi, 
> 
> I have a IBM iDataPlex cluster with QLogic IB QDR switches (QLogic 12200 
> (iDPx) 36-port QDR InfiniBand Switch), currently there are servers with 
> QLogic QME7342 QDR HCA adapters connected to those IB switches ... all 
> is playing nice. 
> 
> Recently I've got a couple of Mellanox IB FDR adapters (MT27500 Family 
> [ConnectX-3]) and I would like them play nicely with existing IB network 
> since I would like to do some research with Mellanox adapters. 
> 
> On last Thursday (5 days ago) I've installed one FDR Mellanox adapter in 
> one of the servers (replaced QLogic QDR adapter) and connected it to the 
> QDR QLogic IB switch. Other servers with QDR QLogic adapters are also 
> connected to the switch and are playing nicely. Mellanox FDR Adapter was 
> detected by the server and OS (centos6), latest ofed installed. After 
> reboot IB diag tools showed link is down, adapter's leds were off ... I 
> didn't have time to play with it so I left it as it was. Yesterday I 
> checked the status once again and I found out that the link is now 
> (State: Active; Physical state: LinkUp; Rate: 40) ... so this looks 
> fine. Today I went ahead and installed adapters in additional two 
> servers and I see the same situation - link is down, adapter's leds were 
> off ... 
> 
> Is it possible that link initialization is taking sooo long (it is now 
> more then one hour since I connected the adapters with the switch)? Is 
> there any "magic" to force the initialization/connection or to persuade 
> Mellanox FDR adapter to go directly to QDR speed? 
> 
> Has anybody got any experiences in mixing QDR and FDR IB, especially 
> connecting Mellanox FDR adapters to QLogic QDR IB switches? Thanks for help! 
> 
> 
_______________________________________________ 
Users mailing list 
Users at lists.openfabrics.org 
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/users 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20130617/bc01df72/attachment.html>