[libfabric-users] Using libfabric on Titan

[Pritchard Jr., Howard]

the GNI provider isn’t supposed on Titan - OS is too old.


-- 
Howard Pritchard

B Schedule
HPC-ENV
Office 9, 2nd floor Research Park
TA-03, Building 4200, Room 03U

Los Alamos National Laboratory





On 5/3/18, 11:21 AM, "Hefty, Sean" <sean.hefty at intel.com> wrote:

>Copying ofiwg, Howard, and Jim.
>
>
>> Apologies if this has been asked before.
>> 
>> I am trying to use the GNI provider for libfabric on Titan, the XK7
>> machine at ORNL. Is this possible/supported?
>> 
>> More specifically, I need to be able to communicate between aprun
>> instances (e.g. aprun ./writer and aprun ./reader are able to
>> communicate with each other) so the sharing of credentials between the
>> programs is necessary. I have been referring to the cray-tests repo as
>> an example of how to use rdma credentials for Cray in Libfabric. I
>> modified this code somewhat by replacing all DRC code with just
>> setting the cookie manually, and had hoped that would work.
>> Unfortunately, the library relies on GNI_GetPtag to set the ptag, and
>> this API call does not seem to work on Titan. I can get the
>> auth_keys_test to pass by adding a ptag field to the
>> fi_gni_raw_auth_key struct along with appropriate language to handle
>> that, but I was wondering if there was some inherently supported way
>> to do that same thing without modifying libfabric.
>> 
>> Other than credentials exchange/loading, i am wondering what I can
>> expect in trying to use the GNI provider on a non-ARIES system. Has
>> anyone done this successfully?
>> 
>> Thanks,
>> Philip