[ofa-general] rdma_create_qp fails with -12

[Or Gerlitz]

Shipman, Galen M. wrote:

> Up to 256 of these are chained together using the next pointer so that a 
> single call to ib_post_send is made for up to a 1MB xfer.
> The number of work requests allocated for the QP is controlled by number 
> of concurrent sends * 256.
> At 16 concurrent sends there is no problem.
> At 64 there is (once we allocate recv work requests as well).

the most problematic aspect in this approach is the memory consumption 
for the --QP-- even when you just get to the 128K limitation of the 
current implementation, when a non-SRQed Lustre server works with 1K 
clients, it consumes 128M of kernel memory for its QPs, its bad.

> It sounds like this can be alleviated by using FMR.

indeed

Or.