[ofa-general] Infiniband Card Trouble

[Tziporet Koren]

Shue, David CTR USAF AFMC AFRL/RITB wrote:
>
> Hello,
>
> I have used the OFED-1.3 software to communicate with the current 
> cards I have. These cards come up as “MT23108” in the logs, and I am 
> not sure whom the manufacturer is. I was able to program the cards, 
> and even install MPICH2 and run tests.
>
> I have recently obtained new IB cards from HP “*HP PCI-X 2-port 4X 
> Fabric (HPC) Adapter” 
> http://h20000.www2.hp.com/bizsupport/TechSupport/Home.jsp?lang=en&cc=id&prodTypeId=12883&prodSeriesId=460713&lang=en&cc=id 
> <http://h20000.www2.hp.com/bizsupport/TechSupport/Home.jsp?lang=en&cc=id&prodTypeId=12883&prodSeriesId=460713&lang=en&cc=id> 
> *and these cards do not work the same. The machine boots up fine with 
> the card in, and shows the card as Mellanox “MT23108” also? The two 
> cards are visibly different in every way. Is the MT23108 a certain 
> platform for IB?
>
Yes it is. Its Mellanox PCIX cards. Maybe you need to upgrade teh FW for 
the new card.
You can get the new FW and burn it using the instruction on Mellanox 
web: http://www.mellanox.com/support/firmware_download.php
Your card is Dual port InfiniHost PCI-X HCA cards (Cougar Cub) 
<http://www.mellanox.com/support/firmware_table_IH.php>

> This is the history of what I did.
>
> 1) Staged the machine RH EL v5
>
> 2) Install the IB card
>
> 3) Boot machine up
>
> 4) Can see the card looking at “lspci” and “dmesg” but nothing in the 
> network area or under “ifconfig” (Just like with the first cards)
>
Can you send output of lspci -vv
>
> 5) I then install the OFED-1.3 software to communicate and configure 
> the card
>
> 6) When I go to start the card (instead of reboot but have tried both 
> ways) /etc/init.d/openib start, it all fails. I then look in the log 
> file and see a bunch of “unknown symbol…” and “disagrees…” for all 
> items of ib_uverbs, ib_umad,iw_cxgb3,ib_path, mlx_ib, and so on.
>
> 7) When I reboot, the machine reaches “UDEV” of the reboot stage, 
> hangs for a little bit, and then many errors show and the machine 
> won’t boot, unless I take the card out. If I uninstall the OFED 
> software, it will reboot fine with the card still in. The card from HP 
> giving me problems, does not appear to have any drivers for it. It 
> looks like HP supports it to work on Windows, and HPUX.
>
What is the machine type you use? Is it IA64?

Tziporet