[ofiwg] GASNet with OFI on PSM provider

[Sayan Ghosh]

Thanks for the clarification. FYI, the PSM conduit of GASNet is no longer
in development (
https://bitbucket.org/berkeleylab/gasnet/src/351130ed5b588c3d05d55a3f56ca0019e8e04758/psm-conduit/README?at=master&fileviewer=file-view-default
).

On Wed, Jun 20, 2018 at 12:08 PM Hefty, Sean <sean.hefty at intel.com> wrote:

> To clarify, I meant GASNet to TrueScale through OFI.  GASNet should have
> a psm-conduit.  OFI targets Omni-Path.
>
>
>
> - Sean
>
>
>
>
>
> This is my understanding as well.  I don’t believe GASNet is a target
> application for TrueScale or PSM.  The high performance interface for
> TrueScale is PSM, rather than verbs, and targets older MPI
> implementations.
>
>
>
> - Sean
>
>
>
>
>
> *From:* ofiwg [mailto:ofiwg-bounces at lists.openfabrics.org
> <ofiwg-bounces at lists.openfabrics.org>] *On Behalf Of *Sayan Ghosh
> *Sent:* Wednesday, June 20, 2018 9:17 AM
> *To:* ofiwg at lists.openfabrics.org
> *Subject:* Re: [ofiwg] GASNet with OFI on PSM provider
>
>
>
> Just wanted to clarify that a discussion with GASNet developers revealed
> that their PSM provider compatibility over OFI is only for PSM2 (since
> TrueScale is an old technology).
>
> https://groups.google.com/a/lbl.gov/forum/#!topic/gasnet-users/1Y_mDmdcCDw
>
>
>
>
> On Tue, Jun 19, 2018 at 10:44 PM Sayan Ghosh <zsayanz at gmail.com> wrote:
>
> I am trying to install GASNet 1.30.0 using OFI on an Intel TrueScale
> network (PSM provider), and although OFI fi_info detects PSM provider
> (see attached output of fi_info), when I mention it during GASNet
> configure (config.log attached, please see line #10126), GASNet warns:
>
>
>
> WARNING: Invalid or unsupported OFI provider specified via the --with-ofi-provider
> flag.
>
> Defaulting to runtime provider feature detection for ofi-conduit.
>
>
>
> GASNet configuration:
>
>
>
> CC=icc CXX=icpc MPI_CC=/home/sghosh/builds/mvapich222-intel-psm/bin/mpicc
> MPI_CFLAGS="-I/home/sghosh/builds/mvapich222-intel-psm/include"
> MPI_LIBS="-L/home/sghosh/builds/mvapich222-intel-psm/lib -lpciaccess -
> lnuma -lxml2" ./configure --prefix=/home/sghosh/builds/gasnet-ofi
> --enable-ofi --with-ofi-home=/home/sghosh/builds/libfabric --disable-psm
> --disable-ibv --with-ofi-spawner=mpi --with-ofi-provider=psm
>
>
>
> GASNet throws this warning during application execution:
>
>
>
> WARNING: Using OFI provider (verbs;ofi_rxm), which has not been validated
> to provide
>
> WARNING: acceptable GASNet performance. You should consider using a more
>
> WARNING: hardware-appropriate GASNet conduit. See ofi-conduit/README.
>
> *** FATAL ERROR: The statically-determined value for
> GASNETC_OFI_HAS_MR_SCALABLE=1 does
>
>   not match the memory registration support that the (verbs;ofi_rxm)
> provider reported.
>
>   This could happen if a provider that previously only supported
> FI_MR_BASIC
>
>   added support for FI_MR_SCALABLE, or if the wrong provider was selected
> at runtime.
>
>   Use configure option --disable-ofi-mr-scalable to correct this.
>
>
>
> Instead of picking up psm, GASNet picks up verbs. I have reported this
> issue on GASNet user list as well. Please let me know if I can try some
> options in building libfabrics for PSM provider, that may fix this issue.
>
>
>
> --
>
> Thanks,
>
> Sayan | https://sg0.github.io/
>
>
>
>
> --
>
> Sayan | https://sg0.github.io/
>


-- 
Sayan | https://sg0.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofiwg/attachments/20180620/8c27a8dc/attachment.html>