[openib-general] Trying to compile mvapich RHEL4U3 for ib.

[Roger Heflin]

I am not having much luck with the default RHEL4U3 setup.  I
appear to have IP over IB running and appearing to work, but
am unable to get any mpi variants to work directly with IB, I
do have it working over tcp with ch_p4.

With mvapich-0.9.7 it errors out in the building
stage with an error ibv_free_device_list/ibv_get_device_list missing,
I cannot find any of the ib libraries on RHEL4U3 that appear to contain
that library.

Using the older mvapich-0.9.6 there is no option to make an ib_gen2
version, and there does not appear to be any ch_gen2 device code.

Using the mvapich-gen2-1.src.rpm from openib.org results in
these errors (on the first thing it tries to compile).
viainit.c: In function `create_cq':
viainit.c:118: error: too few arguments to function `ibv_create_cq'

I have verified that the include file prototype has more arguments, than
are contained in viainit.c.

Trying to use openmpi produced different but still failing results,
everything compiled and linked and HPL would start but never produced
any output from HPL itself, it did produce some things that looked
like internal openmpi errors.

Any suggestion on what I am missing, or if there is another version
that will work?   It looks like there must be alot of API differences
between the different variants that I have.

                                  Roger