[ofiwg] Using libfabric on Titan
sean.hefty at intel.com
Thu May 3 10:21:48 PDT 2018
Copying ofiwg, Howard, and Jim.
> Apologies if this has been asked before.
> I am trying to use the GNI provider for libfabric on Titan, the XK7
> machine at ORNL. Is this possible/supported?
> More specifically, I need to be able to communicate between aprun
> instances (e.g. aprun ./writer and aprun ./reader are able to
> communicate with each other) so the sharing of credentials between the
> programs is necessary. I have been referring to the cray-tests repo as
> an example of how to use rdma credentials for Cray in Libfabric. I
> modified this code somewhat by replacing all DRC code with just
> setting the cookie manually, and had hoped that would work.
> Unfortunately, the library relies on GNI_GetPtag to set the ptag, and
> this API call does not seem to work on Titan. I can get the
> auth_keys_test to pass by adding a ptag field to the
> fi_gni_raw_auth_key struct along with appropriate language to handle
> that, but I was wondering if there was some inherently supported way
> to do that same thing without modifying libfabric.
> Other than credentials exchange/loading, i am wondering what I can
> expect in trying to use the GNI provider on a non-ARIES system. Has
> anyone done this successfully?
More information about the ofiwg