[openfabrics-ewg] OFED-1.0-rc6 is available

Wed Jun 7 23:16:40 PDT 2006

Hi Tziporet:

At the end of install script, we specify IP addresses for ib0 and ib1 interfaces. Then i reboot the system. After the system comes back, i observed that ip address of ib0 interface (ifconfig ib0) is missing. We need to configure again the ip address of ib0 and ib1 interfaces. Please let me know if this is a bug. Why do we need to specify same ip address twice? I can see the IP address which i configure during install time in ifcfg-ib0 file.

I have seen this behaviour on RHEL4 up2 and FC4 servers. Below are some logs.

Thanks
Karun

################################################################

[root at ss23 ~]# /etc/init.d/opensmd status
opensm (pid 3216) is running...
[root at ss23 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ib0
DEVICE=ib0
BOOTPROTO=static
IPADDR=172.20.51.223
NETMASK=255.255.255.0
NETWORK=172.20.0.0
BROADCAST=172.20.51.255
ONBOOT=yes
[root at ss23 ~]# ifconfig ib0
ib0       Link encap:InfiniBand  HWaddr 00:00:00:14:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00  
          inet6 addr: fe80::206:6a00:a000:537/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:2044  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:128 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
[root at ss23 ~]# 

##############################################################################

________________________________

From: openfabrics-ewg-bounces at openib.org on behalf of Tziporet Koren
Sent: Wed 6/7/2006 10:58 AM
To: Tziporet Koren; openfabrics-ewg at openib.org
Cc: openib-general
Subject: [openfabrics-ewg] OFED-1.0-rc6 is available

Hi All,

We have prepared OFED 1.0 RC6.

Release location: https://openib.org/svn/gen2/branches/1.0/ofed/releases

File: OFED-1.0-rc6.tgz

Note: This release is the code freeze release for OFED 1.0. Only showstopper bugs will be fixed.

BUILD_ID:

OFED-1.0-rc6

openib-1.0 (REV=7772)

# User space

https://openib.org/svn/gen2/branches/1.0/src/userspace

# Kernel space

https://openib.org/svn/gen2/branches/1.0/ofed/tags/rc6/linux-kernel

Git:

ref: refs/heads/for-2.6.17

commit d9ec5ad24ce80b7ef69a0717363db661d13aada5

# MPI

mpi_osu-0.9.7-mlx2.1.0.tgz

openmpi-1.1b1-1.src.rpm

mpitests-1.0-0.src.rpm 

OSes:

    * RH EL4 up2: 2.6.9-22.ELsmp

    * RH EL4 up3: 2.6.9-34.ELsmp

    * Fedora C4: 2.6.11-1.1369_FC4

    * SLES10 RC2: 2.6.16.16-1.6-smp

    * SUSE 10 Pro: 2.6.13-15-smp

    * kernel.org: 2.6.16.x

Systems:

    * x86_64

    * x86

    * ia64

    * ppc64

Main changes from RC5:

1.       SDP - libsdp implementation of RFC proposed by Eitan Zahavi; bug fixes in kernel module. See details below.

2.       SRP - bug fixes

3.       Open MPI - new package based on 1.1b1-1

4.       OSU-MPI - See details below.

5.       iSER: Enhanced to support SLES 10 RC1.

6.      IPoIB default configuration changed:

a.       IPoIB configuration at install time is now optional.

b.       The default configuration of IPoIB interfaces (if performed at install time) is DHCP; it can be changed during interactive installation.

c.       For unattended installation one can give a new configuration file. See the example below.

7.       Bug Fixes.

Package limitations:

1.       The ipath driver does not compile/load on most systems. To be fixed in final release. 
Meanwhile, one must work with custom build and not choose ipath driver, or change in the conf file: ib_ipath=n.
I attached a reference ofed-no_ipath.conf file. 
Once Qlogic fixes the backport patches I will publish them on the release page so any one interested can use them with this release.

2.       iSER is working on SuSE SLES 10 RC1 only

IPoIB configuration file example:

If you are going to install OFED on a 32 node cluster and want to use static IPoIB configuration based on Ethernet device configuration follow instructions below:

Assume that the Ethernet IP addresses (eth0 interfaces) of the cluster are: 10.0.0.1 - 10.0.0.32

and you want to assign to ib0 IP addresses in the range: 192.168.0.1 - 192.168.0.32

and to ib1 IP addresses in the range: 172.16.0.1 - 172.16.0.32

Then create the file ofed_net.conf with the following lines:

LAN_INTERFACE_ib0=eth0
IPADDR_ib0=192.168.'*'.'*'
NETMASK_ib0=255.255.0.0
NETWORK_ib0=192.168.0.0
BROADCAST_ib0=192.168.255.255
ONBOOT_ib0=1
LAN_INTERFACE_ib1=eth0
IPADDR_ib1=172.16.'*'.'*'
NETMASK_ib1=255.255.0.0
NETWORK_ib1=172.16.0.0
BROADCAST_ib1=172.16.255.255
ONBOOT_ib1=1

Note: '*' will be replaced by the corresponding octal from the eth0 IP address.

Assuming that you already have OFED configuration file (ofed.conf) with selected packages (created by running OFED-1.0/install.sh)

Run:    ./install.sh -c ofed.conf -net ofed_net.conf

OSU MPI:

*        Added mpi_alltoall fine tuning parameters

*        Added default configuration/documentation file $MPIHOME/etc/mvapich.conf

*        Added shell configuration files  $MPIHOME/etc/mvapich.csh , $MPIHOME/etc/mvapich.csh

*        Default MTU was changed back to 2K for InfiniHost III Ex and InfiniHost III Lx HCAs. For InfiniHost card recommended value is:
VIADEV_DEFAULT_MTU=MTU1024

SDP Details: 

libsdp enhancements according to the RFC:

1.	New config syntax (please see libsdp.conf) 
2.	With no config or empty config use SIMPLE_LIBSDP mode 
3.	Support listening on both tcp and sdp 
4.	Support trying both connections (first SDP then TCP) 
5.	Support IPv4 embedded in IPv6 (also convert back address) 
6.	Comprehensive verbosity logging 
7.	BNF based config parser 

Current SDP limitations:

*        SDP currently does not support sending/receiving out of band data (MSG_OOB).

*        Generally, SDP supports only SOL_SOCKET socket options.

*        The following options can be set but actual support is missing:

o       SO_KEEPALIVE - no keepalives are sent

o       SO_OOBINLINE - out of band data is not supported

o       SDP currently supports setting the following SOL_TCP socket options:

o       TCP_NODELAY, TCP_CORK - but actual support for these options is still missing

*         SDP currently does not handle Zcopy mode messages correctly and does not set MaxAdverts properly in HH/HAH messages.

OFED components tested by Mellanox:

*	Verbs over mthca 
*	IPoIB 
*	OpenSM 
*	OSU-MPI 
*	SRP 
*	SDP 
*	IB administration utils (ibutils) 

Please send us any issues you encounter and/or test results.

Thanks

Tziporet & Vlad

Tziporet Koren

Software Director

Mellanox Technologies

mailto: tziporet at mellanox.co.il <mailto:tziporet at mellanox.co.il> 
Tel +972-4-9097200, ext 380

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20060608/bd0f53c1/attachment.html>