[openib-general] Trying to compile mvapich RHEL4U3 for ib.

[Roger Heflin]

Sayantan Sur wrote:
> Hello Roger,
> 
>> With mvapich-0.9.7 it errors out in the building
>> stage with an error ibv_free_device_list/ibv_get_device_list missing,
>> I cannot find any of the ib libraries on RHEL4U3 that appear to contain
>> that library.
> 
> Thanks for trying out MVAPICH-0.9.7. Currently, we don't have any
> machine with RHEL4U3. We are installing two machines with RHEL4U3 and we
> will try out MVAPICH on that as soon as possible.
> 
> The verbs `ibv_get_device_list' was introduced before the 1.0 branch.
> So, if you have either OpenIB installed from the trunk or from the 1.0
> branch, you _should_ be able to see this verb in the library.
> 
> I am wondering if you are trying out the default versions of the OpenIB
> rpms on RHEL4U3?

Yes, I am trying the default version of RHEL4U3, alot of our
customers would much rather use unmodified RHEL, though I can probably
talk them out of it with a bit of work.   They have some strange
ideas that RHEL is somehow "guaranteed" to work right, and from
what I can tell it won't completely work just because RH did not
include a IB mpi variant, at least not one that I can find.

> 
>> Using the mvapich-gen2-1.src.rpm from openib.org results in
>> these errors (on the first thing it tries to compile).
>> viainit.c: In function `create_cq':
>> viainit.c:118: error: too few arguments to function `ibv_create_cq'
> 
> This is also due to a verb change made a while back to the
> ibv_create_cq. I believe this version of mvapich-gen2 source rpm was
> created against the version of userspace support which is present in the
> very same .src.rpm (you may install those if you want, though they are a
> little old now). The userspace verbs changed after this src rpm was
> created.

> 
>> I have verified that the include file prototype has more arguments, than
>> are contained in viainit.c.
> 
> Yes, it seems that the RPM you have installed is from somewhere in
> between the ibv_create_cq verb change and the later introduction of the
> ibv_get_device list verb.
> 
> I'm wondering if you could try it out with the latest 1.0 branch of
> OpenIB? In addition, we will get back to you asap with our testing on
> RHEL4U3.
> 
> Thanks,
> Sayantan.
> 

Do you know if it would be possible to just replace the userspace
section and not mess with the kernel part of OpenIB?   I am guessing
from what I have read that this is very possible, and only requires
me to remove the already existing RHEL rpms for OpenIB userspace
support.

Thank you very much.

If you guys need access I have 2 test machines that I can give
access to to do whatever testing is needed.

                               Roger