[openib-general] low number of CQ entries allowed

[Sayantan Sur]

Hi,

Today I upgraded Gen2 drivers to revision 2539. I am seeing that there
are CQ & QP creation problems with larger number of CQ entries.

For example, with rdma_lat (part of perftest), I can go upto number of
CQ entries = 256 when I hit an error:

[surs at x5:perftest] ./rdma_lat --tx-depth=256
Couldn't create QP

Upon increasing the CQ entries to slightly larger values, the CQ
creation fails (at exactly 512)

[surs at x5:perftest] ./rdma_lat --tx-depth=512
Couldn't create CQ

I am running these tests on a 3.4 GHz EM64T machine. Has anyone else
also seen these problems?

For applications like MPI, generally the number of CQ entries required
are quite large (since we have only one CQ per MPI process and a lot of
communicating peers). Typical number of CQ entries used (in MVAPICH) is
around 40,000.

Please let me know how to work around this problem.

Thanks,
Sayantan.


-- 
http://www.cse.ohio-state.edu/~surs