[libfabric-users] interoperability between Verbs provider and MPI (MVAPICH)

[Hefty, Sean]

Re-sending because of issues with the OFA email server.  Phil, you probably got this okay, but I don't think the group did.

- Sean

> > Thanks Sean.  I just meant that I have an  MPI program that (in
> > addition to it's MPI calls) tries to use OFI directly to contact an
> > external service.  The MVAPICH library is using libibverbs, not
> > libfabric.  The libfabric library is also using libibverbs.
> 
> Got it.  This makes more sense to me now.
> 
> > I was hoping that maybe there might be a magic environment variable
> > that made MPI and OFI get along on verbs before I dug too far into
> the
> > problem :)  I know that sounds like a random thing to ask about, but
> > the reason that I even thought of it is because the PSM2 provider
> has
> > a FI_PSM2_NAME_SERVER parameter that is is automatically toggled if
> > libfabric detects the presence of OpenMPI or MPICH (see
> > https://ofiwg.github.io/libfabric/v1.5.0/man/fi_psm2.7.html).  I was
> > wondering if the verbs provider did anything similar to subtly
> change
> > it's behavior if it thinks that MPI is being used that would cause
> my
> > OFI test code to work Ok in a standalone program but not within an
> MPI
> > program.
> 
> There's nothing special about verbs that I'm aware of.  I can't think
> of any reason why an app couldn't link to verbs via 2 different
> library paths, though I've never tested with that configuration.
> AFAIK, that should work.  I believe PSM is different in this regard
> and can't support that configuration.  What verbs hardware are you
> using?  It may still be worthwhile to post the question to the linux-
> rdma mailing list as well.
> 
> - Sean