[libfabric-users] fwd: Scatter/Gather send operations

[Hefty, Sean]

Jorn sent the following message to the mailing list, but it was blocked at the server.  I tried to forward it on, but I haven't seen it yet, so I'm forwarding it manually (along with a reply):

- Sean

---
Hi all,

I would like to accelerate my software (NetIO if anybody remembers) by 
using scatter/gather operations like fi_sendv.

I have two questions about that:

1) For the verbs provider it looks like scatter/gather support is very 
limited. By default a vector can only contain up to 4 entries. How far 
can I push that? I would like put there in the order of 100s of entries.

Ultimately, this is limited by hardware.  Each iov entry increases the size of the underlying QP.  Some devices support a value as low as 4, while others can go up to maybe a couple hundred.

I'm not sure how dynamic the verbs provider is in this regard.


2) Concerning memory registrations. All iovec entries would come from an 
address range which can span maybe up to 4 GB. Do I have to register 
every iovec entry, or would be sufficient to do one memory registration 
of the large 4 GB memory region?

You can do a single registration.


Thanks for your help.


Cheers,
Jörn