[openib-general] [mthca] Creation of a SRQ with many WR (> 16K) in kernel level fails

[Or Gerlitz]

Roland Dreier wrote:
>  > anyway, the solution that comes into my mind is to disable creating a
>  > QP/SRQ for which > 128KB allocations are needed. So
>  > mthca_query_device() will set the max_qp_wr and max_srq_wr attributes
>  > to values whose derived size still allows to use kmalloc.
> 
> But that will limit the size of the queues that userspace can create
> too.  I guess we could allocate kernel wrid arrays with vmalloc(), but
> I wonder if anyone actually cares about this limit...

mmm, i would avoid vmalloc if possible. Allocating upto 128K bytes for a 
kernel resource sounds fine.

As for the user space sharing of the same limitation, how about adding 
to the --kernel-- struct ib_device_attr "for user space" buddy fields to 
max_qp_wr max_srq_wr and max_cqe such that each hw driver set both 
values: for the "user space" field the actual hw limitation and for 
"kernel space" field a value which would pass kmalloc.

kernel ULPs calling ibv_device_query would use the original fields, no 
need to patch them. Same for user space ULPs no need to patch them.

However, when the call is made from user space, uverbs_query_device 
copies to the resp struct the "user space" attr.

Or.