[openib-general] segfault on openib mvapich

[Christoph A. Mordasini]

Hi

We are running here mvapich gen 2 downloaded from osu about Sept. 12.,
with 2.6.12.6 from kernel.org, Fedora core 4 (gcc 4.0.0) and the IB tree
from openib.org downloaded about 3 weeks ago, without any subsequent
patches added. 

The hardware of the cluster is somewhat special: We use AMD dual core
Athlons on a ASUS A8N-E board, with Mellanox MHEL-CF256-T HCA (PCIe x8)
in the PCIe x16 ("graphics") slot. The idea to use standard customer
boards (not server) with a pcie x16 "graphics" slot  for IB comes from
Don Holmgreen at Fermilab and is a great way to build inexpensive
clusters with dual core nodes.  

We had a number of problems before we could make mvapich work, but with
the help of osu, it now works perfectly. 

We also had inexplicable segfaults with different, very simple mpi
programs. We finally found out that these went away after changing the
following things for the CFLAGS in the mvapich make file (e.g.
mvapich.make.gcc)

1) delete -DLAZY_MEM_UNREGISTER
2) use -O2 instead of -O3 
(not sure if the second point also matters)

This will probably have some negative performance impact, which I
haven't tried to quantify.

I just saw that your problem was due to LD_LIBRARY_PATH (and not to the
compilation options), but maybe this will help someone else.

By the way, I have the following question: Is there a more mvapich
related newsgroup? 

Thanks and kind regards

Chris


-- 
************************************************
*                                              *
*           Christoph A. Mordasini             *
*                                              *
*    Theoretical Astrophysics Research Group   *
*                                              *
*          Physikalisches Institut             *
*             University of Bern               *
*                                              *
*           Phone: +41316314409                *
*  e-Mail: christoph.mordasini at phim.unibe.ch   *
*                                              *
************************************************