[ofa-general] ib_create_qp and ib_get_err_str weirdness

[Diego Guella]

Hello,

I am using Mellanox WinOF 2.0.0 with a MHES14-XTC SDR single-port card.
I noticed a strange behavior of ib_create_qp function:

-----
memset(&qp_create, 0, sizeof(qp_create));
qp_create.qp_type = IB_QPT_RELIABLE_CONN; // Reliable Connected
qp_create.sq_depth = ctx->qdepth;
qp_create.rq_depth = ctx->qdepth;
qp_create.sq_sge = ctx->hca_attr->max_sges;
qp_create.rq_sge = ctx->hca_attr->max_sges;
qp_create.h_sq_cq = ctx->cq_h;
qp_create.h_rq_cq = ctx->cq_h;
qp_create.h_srq = NULL;
qp_create.sq_signaled = 1;
ctx->qp_h = 0;
rc = ib_create_qp(ctx->pd_h, &qp_create, NULL, NULL, &ctx->qp_h);
-----
return value ("rc") is 3 (=IB_INVALID_PARAMETER).

I spent some time figuring out the problem was the SQ SGE value:
http://lists.openfabrics.org/pipermail/general/2006-June/023417.html

According to iba/ib_al.h:
-----
* IB_INVALID_MAX_SGE
* The requested maximum number of scatter-gather entries for the send or
* receive queue could not be supported.
-----
so, why the return value isn't 22 (=IB_INVALID_MAX_SGE)?

In the discussion I mentioned, it turned out that even using 
hca_attr->max_sges there is the possibility that ib_create_qp fails.
Which is my case.
I have the need to send some audio buffers (32 or more) from an IO node to a 
computing node using RDMA WRITE.
The ownership of the buffers is of the audio driver, and I haven't the 
guarantee that the audio buffers are contiguous.
I was trying and send them using the lowest possible number of WR, each one 
with the highest possible number of sge.
But, given the hca_attr->max_sge unreliability, how do you recommend to 
achieve this goal?
Should I post a WR for each buffer I'd want to send through RDMA WRITE?


Another less-related problem:
ib_get_err_str is not correct for every input value, for example I noticed 
that for
ib_get_err_str(IB_INVALID_PD_HANDLE) the string returned is 
IB_INVALID_MR_HANDLE


I don't know if these problems apply to linux too, so I'm including general 
list.

Thanks and best regards,
Diego