[libfabric-users] FI_MR_SCALABLE on GNI

[Howard Pritchard]

Hi John,

Interesting you ask about this.  The answer is that the matrix is only
partially correct.  The Aries hardware doesn't really support Mr. Scalable
- it supports a very constrained sort of Mr. Scalable using
FI_MR_MMU_NOTIFY.
There are other problems with the way it was implemented as well.  For GNI
provider, you'll do better sticking with FI_MR_BASIC.  See the patches
we're applying to Open MPI for this issue.


See https://github.com/ofiwg/libfabric/issues/6194

Howard


Am Mo., 31. Aug. 2020 um 05:00 Uhr schrieb Biddiscombe, John A. <
biddisco at cscs.ch>:

> Dear List
>
> libfabric:16321:gni:core:ofi_check_mr_mode():545<info> Invalid memory
> registration mode
> libfabric:16321:gni:core:ofi_check_mr_mode():546<info> Expected:
> FI_MR_BASIC, FI_MR_MMU_NOTIFY
> libfabric:16321:gni:core:ofi_check_mr_mode():546<info> Given:
> FI_MR_SCALABLE
> libfabric:16321:gni:domain:_gnix_ep_getinfo():597<info> [16321:1] failed
> ofi_check_mr_mode, ret=-61
>
>
> I'm using the libfabric 1.11 tag from the git repo and according to the
> feature matrix, scalable memory registration is supported on gni, however,
> I get the above error in the logs when I ask for it in using
>
>     fabric_hints_->domain_attr->mr_mode = FI_MR_SCALABLE;
>
>
> Is the matrix incorrect, or should I keep playing with flags in case I've
> messed something up elsewhere?
>
>
> Thanks
>
>
> JB
> _______________________________________________
> Libfabric-users mailing list
> Libfabric-users at lists.openfabrics.org
> https://lists.openfabrics.org/mailman/listinfo/libfabric-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20200831/27b7efbe/attachment.htm>