[ofa-general] Infiniband Problems

[Roland Dreier]

 > 1. Sometimes observe RDMA data transfer stalls of ~ 1.0 second

Could it be an RNR NAK?  You didn't really describe your protocol, but
if you use send operations and if you do a send without a matching
receive on the other side, then you might end up stalling the QP for a
while.

 > 2. Creation of a Queue Pair is rejected when I have mapped a region of
 > memory greater than about 1.35GB.

I don't really understand this problem.  Are you able to map more
memory, and then ibv_create_qp() fails if you do?  Later you say

 > Ideally, we would like the to be able to write anywhere within a 2GB
 > (or larger) shared memory segment. However, when I attempt to do this,
 > the call to fails with REJ.

You didn't say which call fails with REJ, and I'm not even sure I
understand what it means to "fail with REJ".

On x86-64, the limit on how much memory you can register should be
much higher, closer to 32 GB by default.

 - R.