[ewg] OFED 3.2 - Errors with admin-rdma script and make on x86

[Daniel A. Vega]

Hi Vladimir

I managed to get OFED 3.2 built and installed on ppc64.
I also was able to IPoIB to work as well!

System A:

  # ibv_devinfo
hca_id: mlx4_0
         transport:                      InfiniBand (0)
         fw_ver:                         2.6.000
         node_guid:                      0002:c903:0001:28ec
         sys_image_guid:                 0002:c903:0001:28ef
         vendor_id:                      0x02c9
         vendor_part_id:                 25418
         hw_ver:                         0xA0
         board_id:                       IBM08A0000001
         phys_port_cnt:                  2
                 port:   1
                         state:                  PORT_ACTIVE (4)
                         max_mtu:                2048 (4)
                         active_mtu:             2048 (4)
                         sm_lid:                 1
                         port_lid:               2
                         port_lmc:               0x00
                         link_layer:             InfiniBand

                 port:   2
                         state:                  PORT_DOWN (1)
                         max_mtu:                2048 (4)
                         active_mtu:             2048 (4)
                         sm_lid:                 0
                         port_lid:               0
                         port_lmc:               0x00
                         link_layer:             InfiniBand

# ip addr show ib0
4: ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc pfifo_fast 
state UP qlen 256
     link/infiniband 
80:00:00:48:fe:80:00:00:00:00:00:00:00:02:c9:03:00:01:28:ed brd 
00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
     inet 192.168.0.198/23 brd 192.168.0.255 scope global ib0
     inet6 fe80::202:c903:1:28ed/64 scope link
        valid_lft forever preferred_lft forever

# ping -c 3 192.168.0.199
PING 192.168.0.199 (192.168.0.199) 56(84) bytes of data.
64 bytes from 192.168.0.199: icmp_seq=1 ttl=64 time=5.26 ms
64 bytes from 192.168.0.199: icmp_seq=2 ttl=64 time=0.055 ms
64 bytes from 192.168.0.199: icmp_seq=3 ttl=64 time=0.047 ms

System B:

# ibv_devinfo
hca_id: mlx4_0
         transport:                      InfiniBand (0)
         fw_ver:                         2.6.000
         node_guid:                      0002:c903:0001:0458
         sys_image_guid:                 0002:c903:0001:045b
         vendor_id:                      0x02c9
         vendor_part_id:                 25418
         hw_ver:                         0xA0
         board_id:                       IBM08A0000001
         phys_port_cnt:                  2
                 port:   1
                         state:                  PORT_ACTIVE (4)
                         max_mtu:                2048 (4)
                         active_mtu:             2048 (4)
                         sm_lid:                 1
                         port_lid:               3
                         port_lmc:               0x00
                         link_layer:             InfiniBand

                 port:   2
                         state:                  PORT_DOWN (1)
                         max_mtu:                2048 (4)
                         active_mtu:             2048 (4)
                         sm_lid:                 0
                         port_lid:               0
                         port_lmc:               0x00
                         link_layer:             InfiniBand

# ip addr show ib0
4: ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc pfifo_fast 
state UP qlen 256
     link/infiniband 
80:00:00:48:fe:80:00:00:00:00:00:00:00:02:c9:03:00:01:04:59 brd 
00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
     inet 192.168.0.199/23 brd 192.168.0.255 scope global ib0
     inet6 fe80::202:c903:1:459/64 scope link
        valid_lft forever preferred_lft forever

# ping -c 3 192.168.0.199
PING 192.168.0.199 (192.168.0.199) 56(84) bytes of data.
64 bytes from 192.168.0.199: icmp_seq=1 ttl=64 time=0.028 ms
64 bytes from 192.168.0.199: icmp_seq=2 ttl=64 time=0.010 ms
64 bytes from 192.168.0.199: icmp_seq=3 ttl=64 time=0.007 ms


On 3/21/2012 10:54 AM, Woodruff, Robert J wrote:
> Vlad wrote,
>> >Hi Pradeep,
>> >You can use OFED-3.2 build to get the user space and kernel:>http://www.openfabrics.org/downloads/OFED/ofed-3.2-daily/OFED-3.2-20120223-0313.tgz
> I tried this new package on a couple of systems in my lab and after disabling
> the chelsio adapter in the ofed.conf file, I was able to get it to successfully
> build on a SLES 11 SP 2 system and was able to get Intel MPI to run over uDAPL.
>
> However, for some reason I cannot seem to get IPoIB to work and I am not sure
> if it is a S/W problem or something with my H/W. Has anyone gotten IPoIB to
> work from the OFED-3.2 daily build package ?


-- 
Daniel A. Vega
Dept. 7UEA/7T IBM LTC RDMA Fabrics Development
Ely, MN 55731
Phone: 218-235-0977
Email: dvega at linux.vnet.ibm.com