[libfabric-users] fi_read verbs ENODATA

[Arne Struck]

Am 23.10.2020 um 18:30 schrieb Hefty, Sean:
>> hints->caps = FI_RMA|FI_MSG;
>>
>> hints->ep_attr->type = FI_EP_MSG;
>>
>> hints->addr_format = FI_SOCKADDR_IN;
>>
>> hints->fabric_attr->prov_name = g_strdup("verbs");
>>
>> hints->mode = FI_LOCAL_MR;
> For verbs and any recent release of libfabric, you will want to remove setting this.  Instead, set:
>
> hints->domain_attr->mr_mode = FI_MR_LOCAL | FI_MR_VIRT_ADDR |
> 					FI_MR_ALLOCATED | FI_MR_PROV_KEY;

Ah ok, I thought since FI_BASIC is no longer around, the provider did 
not need those assurances.

With "For verbs and any recent release of libfabric, you will want to 
remove setting this" you are referring to hints->mode = FI_LOCAL_MR and 
not the other settings? How would you otherwise specify a specific 
provider or the address format used?

>>       mode: [ FI_RX_CQ_DATA ]
> You will want to add this to hints->mode if you plan on using CQ data (verbs refers to this as immediate data).  You can leave this off if not.
>   

Thanks for the hint :).

>
> The problem is that verbs requires additional support by the application, mostly to handle memory registration restrictions.  Those are the mr_mode bits called out above.

The application should support those features (with one specific 
exception 1 exception where I still have to register the local memory 
region).


Thanks for the quick answer,

Greetings, Arne.