[libfabric-users] verbs provider scatter-gather

[Hefty, Sean]

> Modes
> Verbs provider requires applications to support the following modes:
> 
> FI_EP_MSG endpoint type
>     - FI_LOCAL_MR / FI_MR_LOCAL mr mode.
> 
> Or am I missing something?

Hmm... I thought with the cache enabled, registration could be skipped.  It should be possible.  But looking at the code, I don't see support for this at the verbs layer.  I think only going through the RxM path (RDM endpoint) avoids needing to register.

> So I think I must register a memory regions for each entry in the iovec by calling
> fi_mr_reg, and passing the memory regions to fi_recvv.  Or I could call libibverbs
> ibv_reg_mr, construct a WR and SGL, and call ibv_post_recv.  But that discards a big
> advantage of libfabric, namely abstracting the providers.  Am I on the right track
> here?

The first option will work.  You need to register each iovec and pass the result to fi_sendv/fi_recvv.

- Sean