[mvapich-discuss] RE: [openfabrics-ewg] Current OFED kernel snapshot

[Sayantan Sur]

Hi,

* On May,10 Roland Dreier<rdreier at cisco.com> wrote :
>     Abhinav> By referring to the PPC64 architecture, i was mentioning
>     Abhinav> about the IBM HCAs(4x/12x) running on GX/GX+ Bus. To the
>     Abhinav> best of my knowledge, these HCAs do not support the
>     Abhinav> features mentioned above.
> 
> Hmm, making this a compile-time thing seems like a problem then.  Some
> ppc64 systems have IBM eHCAs and some have Mellanox and/or PathScale
> HCAs.  Shouldn't the same MPI package work on all of these systems?

We've just incorporated an autodetection utility as a part of our build
script (make.mvapich.gen2). It essentially reads the type of HCA from
the standard location /sys/class/infiniband/<hca>/hca_type. Using this
script, we have decoupled the choice of architecture and the InfiniBand
card in the system. We believe using this script, MVAPICH will be able
to work on PPC64 systems connected with Mellanox/Pathscale cards.

On our installation of the IBM PPC64 systems, there is no `hca_type'
file present. If that file is there, then the above mentioned
"decoupling" thing can be done easily by trivially modifying our script.
Does anyone have a PPC64 installation which has that file?

With OpenIB/Gen2 standardized method of exporting HCA types, we will
soon have runtime detection and optimization.

Thanks,
Sayantan.

-- 
http://www.cse.ohio-state.edu/~surs