[openib-general] libmthca problem: max_inline_size

[Michael S. Tsirkin]

Hello, Roland!
Currently, libmthca calculates the qp capability max_inline_size based
on the wqe size (wqe_shift) field.
However, especially for memfree HCAs, the maximum descriptor size supported
by the HCA may be less than the full wqe size (this is reported in the
max_desc_sz or max_desc_sz_sq/max_desc_sz_rq), and as a result the max inline
size returned to the caller is too large.

I see three possible solutions:
1. Do not increase max_inline_size beyond what is given by the user.
   This is unfortunate bigger inline size may provide optimization opportunity.
2. Return the actual QP capability in create qp command.
   This is an ABI change, although the library can be made to work in a
   backward compatible way.
3. Add a command (device specific) to query the max descriptor size supported
   by the HCA (returned by query dev lim) and calculate max_inline_size
   based on that.
   Again, this is an ABI change.

I am inclining towards the second option (2.) since this way the
resulting capability calculations will be all in one place in kernel.

There's a similiar problem in mthca_arbel_post_send where the
inline data size is checked against the expression

	int max_size = (1 << qp->sq.wqe_shift) - sizeof *seg - size * 16;

I would imagine the way to fix that is to add a max_inline_size field
to the mthca_qp structure and is that instead of 1 << qp->sq.wqe_shift.

Pls let me know what do you think.
Thanks,

-- 
MST