[ofa-general] max inline data query ?

[Or Gerlitz]

Dotan Barak wrote:
> If one wishes to use maximum inline data, he should open a QP with 
> maximum number of s/g entries in the QP->SQ
> (ibv_qp_init_attr.cap.max_send_sge). The actual number of bytes which 
> can be send as inline will be returned in the
> attribute ibv_qp_init_attr.cap.max_inline_data after ibv_create_qp 
> will be finished.
>
Its correct that the maximal inline length can be get this way, but 
please note that the inline size as a function of the send s/g entries 
has "steps" so for some values of n=|s/g|  f(n+1) =  f(n)  but n+1 would 
consume more memory.

Basically, inline sending tends to improve latency (only one DMA 
transaction, to fetch the descriptor) but it increases the memory 
consumption and the CPU utilization (since the user space hw library 
does a copy). My take here is that it should be used with care, going 
simple you can create the QP with the s/g count you need per your design 
and do inline send only when possible under this count, etc.

Or.