[libfabric-users] Using libfabric on Titan

[Philip Davis]

Hello,

Apologies if this has been asked before.

I am trying to use the GNI provider for libfabric on Titan, the XK7 machine at ORNL. Is this possible/supported?

More specifically, I need to be able to communicate between aprun instances (e.g. aprun ./writer and aprun ./reader are able to communicate with each other) so the sharing of credentials between the programs is necessary. I have been referring to the cray-tests repo as an example of how to use rdma credentials for Cray in Libfabric. I modified this code somewhat by replacing all DRC code with just setting the cookie manually, and had hoped that would work. Unfortunately, the library relies on GNI_GetPtag to set the ptag, and this API call does not seem to work on Titan. I can get the auth_keys_test to pass by adding a ptag field to the fi_gni_raw_auth_key struct along with appropriate language to handle that, but I was wondering if there was some inherently supported way to do that same thing without modifying libfabric.

Other than credentials exchange/loading, i am wondering what I can expect in trying to use the GNI provider on a non-ARIES system. Has anyone done this successfully?

Thanks,
Philip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20180503/fdda483b/attachment.html>