[libfabric-users] verbs provider scatter-gather

[Hefty, Sean]

> vrb_mr_regv returns FI_EINVAL if count (the number of entries in the iovec) is larger
> than 1:
> 
> 
> 
>                 if (OFI_UNLIKELY(count > 1))
> 
>                                 return -FI_EINVAL;
> 
> 
> 
> To the best of my knowledge, libibverbs supports scatter-gather, so why is this check
> present?  Did I miss some configuration that redefines the OFI_ UNLIKELY macro?

Verbs does not support registering a SGL.  It can used SGLs for send/receive operations, but can only register a single iovec.  For reference, this is the verbs registration call:

struct ibv_mr *ibv_reg_mr(struct ibv_pd *pd, void *addr, size_t length,
			  int access);

Registration of an iovec would likely be difficult for HW to support without imposing some restrictions (e.g. each SGE must be aligned and a multiple of a page size).

- Sean