[Users] Config errors in IBA7322 QDR InfiniBand HCA
German Anders
ganders at despegar.com
Fri Oct 9 11:46:53 PDT 2015
Ok so I get to the point where I can see from the IB network the HP BLc 4X
QDR switch, but.. the 'internal' switch can't see any of the blades HCA,
also the ports on the blades still report a Down State and a Physical
Polling state.
# ibstat
CA 'qib0'
CA type: InfiniPath_QMH7342
Number of ports: 2
Firmware version:
Hardware version: 2
Node GUID: 0x0011750000791fec
System image GUID: 0x0011750000791fec
Port 1:
State: Down
Physical state: Polling
Rate: 40
Base lid: 65535
LMC: 0
SM lid: 65535
Capability mask: 0x0761086a
Port GUID: 0x0011750000791fec
Link layer: InfiniBand
Port 2:
State: Down
Physical state: Polling
Rate: 40
Base lid: 65535
LMC: 0
SM lid: 65535
Capability mask: 0x0761086a
Port GUID: 0x0011750000791fed
Link layer: InfiniBand
GWIB01: *(LID 24) *-> SX6036G
Infiniscale-IV* M*ellanox Technologies: *(LID 29)* -> QLogic HP BLc 4X QDR
Switch
GWIB02: *(LID 23)* -> SX6036G
SWIB01 *(LID 2) *-> SX6018F
SWIB02: *(LID 1)* -> SX6018F
SM running:
SWIB01: with priority 8
SWIB02: with priority 7
find attach a pdf with the diagram (any other 'best' way to cable them?)
advises?
# iblinkinfo
CA: MF0;GWIB01:SX60XX/GW:
0xe41d2d030031e9c2 25 1[ ] ==( 4X 14.0625 Gbps Active/
LinkUp)==> 24 37[ ] "MF0;GWIB01:SX6036G/U1" ( )
*Switch: 0x0002c902004b0918 Infiniscale-IV Mellanox Technologies:*
29 1[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
24 9[ ] "MF0;GWIB01:SX6036G/U1" ( )
29 2[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
23 9[ ] "MF0;GWIB02:SX6036G/U1" ( )
29 3[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 4[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 5[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 6[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 7[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 8[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 9[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 10[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 11[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 12[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 13[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 14[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 15[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 16[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 17[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 18[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 19[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 20[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 21[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 22[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 23[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 24[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 25[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 26[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 27[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 28[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 29[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 30[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 31[ ] ==( Down/ Polling)==> [ ]
"" ( )
29 32[ ] ==( Down/ Polling)==> [ ]
"" ( )
CA: MF0;GWIB02:SX60XX/GW:
0xe41d2d030031eb42 28 1[ ] ==( 4X 14.0625 Gbps Active/
LinkUp)==> 23 37[ ] "MF0;GWIB02:SX6036G/U1" ( )
*Switch: 0xe41d2d030031e9c1 MF0;GWIB01:SX6036G/U1:*
24 1[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 2[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 3[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 4[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 5[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 6[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 7[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 8[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 9[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
29 1[ ] "Infiniscale-IV Mellanox Technologies" ( )
24 10[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 11[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 12[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 13[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 14[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 15[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 16[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 17[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 18[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 19[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 20[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 21[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 22[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 23[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 24[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 25[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 26[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 27[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 28[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 29[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 30[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 31[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 32[ ] ==( Down/ Polling)==> [ ]
"" ( )
24 33[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
23 33[ ] "MF0;GWIB02:SX6036G/U1" ( Could be 14.0625 Gbps)
24 34[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
1 16[ ] "MF0;SWIB02:SX6018/U1" ( Could be 14.0625 Gbps)
24 35[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
23 35[ ] "MF0;GWIB02:SX6036G/U1" ( Could be 14.0625 Gbps)
24 36[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
2 18[ ] "MF0;SWIB01:SX6018/U1" ( )
24 37[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
25 1[ ] "MF0;GWIB01:SX60XX/GW" ( )
*Switch: 0xf45214030073f500 MF0;SWIB02:SX6018/U1:*
1 1[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
13 2[ ] "cibmon01 HCA-1" ( )
1 2[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
14 2[ ] "cibmon02 HCA-1" ( )
1 3[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
12 2[ ] "cibmon03 HCA-1" ( )
1 4[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
20 2[ ] "cibosd01 HCA-1" ( )
1 5[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
17 2[ ] "cibosd02 HCA-1" ( )
1 6[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
22 2[ ] "cibosd03 HCA-1" ( )
1 7[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
18 2[ ] "cibosd04 HCA-1" ( )
1 8[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
7 2[ ] "cibosd05 HCA-1" ( )
1 9[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
27 2[ ] "cibosd06 HCA-1" ( )
1 10[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
15 2[ ] "cibosd07 HCA-1" ( )
1 11[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
6 2[ ] "cibosd08 HCA-1" ( )
1 12[ ] ==( Down/ Polling)==> [ ]
"" ( )
1 13[ ] ==( Down/ Polling)==> [ ]
"" ( )
1 14[ ] ==( Down/ Polling)==> [ ]
"" ( )
1 15[ ] ==( Down/ Polling)==> [ ]
"" ( )
1 16[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
24 34[ ] "MF0;GWIB01:SX6036G/U1" ( Could be 14.0625 Gbps)
1 17[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
2 17[ ] "MF0;SWIB01:SX6018/U1" ( )
1 18[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
23 36[ ] "MF0;GWIB02:SX6036G/U1" ( Could be 14.0625 Gbps)
*Switch: 0xe41d2d030031eb41 MF0;GWIB02:SX6036G/U1:*
23 1[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 2[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 3[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 4[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 5[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 6[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 7[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 8[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 9[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
29 2[ ] "Infiniscale-IV Mellanox Technologies" ( )
23 10[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 11[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 12[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 13[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 14[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 15[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 16[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 17[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 18[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 19[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 20[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 21[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 22[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 23[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 24[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 25[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 26[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 27[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 28[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 29[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 30[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 31[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 32[ ] ==( Down/ Polling)==> [ ]
"" ( )
23 33[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
24 33[ ] "MF0;GWIB01:SX6036G/U1" ( Could be 14.0625 Gbps)
23 34[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
2 16[ ] "MF0;SWIB01:SX6018/U1" ( Could be 14.0625 Gbps)
23 35[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
24 35[ ] "MF0;GWIB01:SX6036G/U1" ( Could be 14.0625 Gbps)
23 36[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
1 18[ ] "MF0;SWIB02:SX6018/U1" ( Could be 14.0625 Gbps)
23 37[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
28 1[ ] "MF0;GWIB02:SX60XX/GW" ( )
*Switch: 0xe41d2d0300097630 MF0;SWIB01:SX6018/U1:*
2 1[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
4 1[ ] "cibmon01 HCA-1" ( )
2 2[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
5 1[ ] "cibmon02 HCA-1" ( )
2 3[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
3 1[ ] "cibmon03 HCA-1" ( )
2 4[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
19 1[ ] "cibosd01 HCA-1" ( )
2 5[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
10 1[ ] "cibosd02 HCA-1" ( )
2 6[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
21 1[ ] "cibosd03 HCA-1" ( )
2 7[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
11 1[ ] "cibosd04 HCA-1" ( )
2 8[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
16 1[ ] "cibosd05 HCA-1" ( )
2 9[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
26 1[ ] "cibosd06 HCA-1" ( )
2 10[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
8 1[ ] "cibosd07 HCA-1" ( )
2 11[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==>
9 1[ ] "cibosd08 HCA-1" ( )
2 12[ ] ==( Down/ Polling)==> [ ]
"" ( )
2 13[ ] ==( Down/ Polling)==> [ ]
"" ( )
2 14[ ] ==( Down/ Polling)==> [ ]
"" ( )
2 15[ ] ==( Down/ Polling)==> [ ]
"" ( )
2 16[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
23 34[ ] "MF0;GWIB02:SX6036G/U1" ( Could be 14.0625 Gbps)
2 17[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
1 17[ ] "MF0;SWIB02:SX6018/U1" ( )
2 18[ ] ==( 4X 10.0 Gbps (FDR10) Active/ LinkUp)==>
24 36[ ] "MF0;GWIB01:SX6036G/U1" ( )
# lsmod |grep ib
ib_ucm 18450 0
ib_uverbs 46826 1 ib_ucm
ib_ipoib 77741 0
ib_cm 42689 2 ib_ucm,ib_ipoib
ib_sa 33950 2 ib_cm,ib_ipoib
ib_umad 22129 8
ib_mthca 129035 0
ib_qib 390078 1
ib_mad 47486 5 ib_cm,ib_sa,ib_qib,ib_umad,ib_mthca
ib_core 88311 9
ib_cm,ib_sa,ib_mad,ib_qib,ib_ucm,ib_umad,ib_uverbs,ib_ipoib,ib_mthca
ib_addr 18923 1 ib_core
dca 15130 1 ib_qib
libiscsi 57233 1 be2iscsi
libahci 32190 1 ahci
scsi_transport_iscsi 99909 2 be2iscsi,libiscsi
any ideas? I'm really new to ib and can't get the qlogic hp blc 4x qdr ib
connectivity,.. don't know if i need to configure anything else or maybe
there's a problem with the backplane of the encl,... i really don't know,
and sorry if someone is from hp, but hp does not had the 'best' service out
there..specially from ib perspective..
thanks in advance,
cheers,
*German* <ganders at despegar.com>
2015-10-07 1:56 GMT-03:00 Weiny, Ira <ira.weiny at intel.com>:
> I don’t know if Ubuntu includes the ipath verbs packages or not. I would
> assume so.
>
>
>
> If not the ipathverbs packages are available here:
>
>
>
> libipathverbs: Mike Marciniszyn
>
> http://www.openfabrics.org/downloads/libipathverbs
>
> git://github.com/01org/libipathverbs.git master
>
> (infinipath at intel.com<mailto:infinipath at intel.com>)
>
>
>
> Also, if your applications can use it, you will get better performance
> using PSM which is available here:
>
>
>
> infinipath-psm: CQ Tang
>
> (infinipath at intel.com<mailto:infinipath at intel.com>)
>
> http://www.openfabrics.org/downloads/infinipath-psm
>
>
>
>
>
> Ira
>
>
>
>
>
> *From:* users-bounces at lists.openfabrics.org [mailto:
> users-bounces at lists.openfabrics.org] *On Behalf Of *German Anders
> *Sent:* Tuesday, October 06, 2015 12:32 PM
> *To:* Hal Rosenstock
> *Cc:* users at lists.openfabrics.org
> *Subject:* Re: [Users] Config errors in IBA7322 QDR InfiniBand HCA
>
>
>
> Hi Hal,
>
>
> Thanks a lot for the quick response. I'll try those commands and let you
> know. Regarding the software, is there any 'best soft' to use with Ubuntu
> and this kind of adapter? Qlogic
>
> Thanks in advance,
>
> Best regards,
>
>
> *German*
>
>
>
> 2015-10-06 16:30 GMT-03:00 Hal Rosenstock <hal.rosenstock at gmail.com>:
>
> Hi German,
>
>
>
> You can't get at the switches because the local ports are disabled.
>
>
>
> Start with enabling these ports. Something like:
>
>
>
> ibportstate -D 0 1 enable
>
> ibportstate -D 0 2 enable
>
>
>
> Hopefully that should move the ports out of disabled physical state and if
> so try ibswitches.
>
>
>
> You appear to be using MOFED distribution. Is there a reason for that ?
> I'm not sure that it includes packages for libipathverbs or libcxgb4.
>
>
>
> -- Hal
>
>
>
>
>
> On Tue, Oct 6, 2015 at 2:48 PM, German Anders <ganders at despegar.com>
> wrote:
>
> Hi All,
>
> I'm in the middle of a new Infiniband implementation but I'm getting some
> errors, the basic setup is a HP Enclosure with a IB SW (Infiniscale-IV
> Mellanox Technologies), then I've setup a Blade with Ubuntu 14.04 and when
> trying to configure the IB support I'm getting some errors:
>
> # lscpi
> (...)
> *41:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)*
>
>
> # ibstatus
> Infiniband device 'qib0' port 1 status:
> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff8
> base lid: 0xffff
> sm lid: 0xffff
> state: 1: DOWN
> phys state: 3: Disabled
> rate: 40 Gb/sec (4X QDR)
> link_layer: InfiniBand
>
> Infiniband device 'qib0' port 2 status:
> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff9
> base lid: 0xffff
> sm lid: 0xffff
> state: 1: DOWN
> phys state: 3: Disabled
> rate: 40 Gb/sec (4X QDR)
> link_layer: InfiniBand
>
> root at e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64# ibstat
> CA 'qib0'
> CA type: InfiniPath_QMH7342
> Number of ports: 2
> Firmware version:
> Hardware version: 2
> Node GUID: 0x0011750000791ff8
> System image GUID: 0x0011750000791ff8
> Port 1:
> State: Down
> Physical state: Disabled
> Rate: 40
> Base lid: 65535
> LMC: 0
> SM lid: 65535
> Capability mask: 0x07610868
> Port GUID: 0x0011750000791ff8
> Link layer: InfiniBand
> Port 2:
> State: Down
> Physical state: Disabled
> Rate: 40
> Base lid: 65535
> LMC: 0
> SM lid: 65535
> Capability mask: 0x07610868
> Port GUID: 0x0011750000791ff9
> Link layer: InfiniBand
>
>
> # ibstatus
> Infiniband device 'qib0' port 1 status:
> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff8
> base lid: 0xffff
> sm lid: 0xffff
> state: 1: DOWN
> phys state: 3: Disabled
> rate: 40 Gb/sec (4X QDR)
> link_layer: InfiniBand
>
> Infiniband device 'qib0' port 2 status:
> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff9
> base lid: 0xffff
> sm lid: 0xffff
> state: 1: DOWN
> phys state: 3: Disabled
> rate: 40 Gb/sec (4X QDR)
> link_layer: InfiniBand
>
> Some of the errors besides that the phys state of the ports is disabled:
>
> # ibswitches
> src/query_smp.c:228; can't open UMAD port ((null):0)
> /usr/sbin/ibnetdiscover: iberror: failed: discover failed
>
> # ibv_devinfo
> libibverbs: Warning: couldn't load driver 'ipathverbs':
> libipathverbs-rdmav2.so: cannot open shared object file: No such file or
> directory
> libibverbs: Warning: couldn't load driver 'cxgb4': libcxgb4-rdmav2.so:
> cannot open shared object file: No such file or directory
> hca_id: qib0
> transport: InfiniBand (0)
> fw_ver: 0.0.0
> node_guid: 0011:7500:0079:1ff8
> sys_image_guid: 0011:7500:0079:1ff8
> vendor_id: 0x1175
> vendor_part_id: 29474
> hw_ver: 0x2
> board_id: InfiniPath_QMH7342
> phys_port_cnt: 2
> port: 1
> state: PORT_DOWN (1)
> max_mtu: 4096 (5)
> active_mtu: 4096 (5)
> sm_lid: 65535
> port_lid: 65535
> port_lmc: 0x00
> link_layer: InfiniBand
>
> port: 2
> state: PORT_DOWN (1)
> max_mtu: 4096 (5)
> active_mtu: 4096 (5)
> sm_lid: 65535
> port_lid: 65535
> port_lmc: 0x00
> link_layer: InfiniBand
>
> Also, In what packages are the *ipathverbs* and *cxgb4* contained? Also,
> does anyone had any experience with HP IB switches?
>
> Thanks in advance,
>
>
>
> *German*
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.openfabrics.org
> http://lists.openfabrics.org/mailman/listinfo/users
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20151009/2140947f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Infiniband Diagram v1.pdf
Type: application/pdf
Size: 237019 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20151009/2140947f/attachment.pdf>
More information about the Users
mailing list