[ofa-general] ibv_modify_qp() return value 22

[Joerg Zinke]

Hi,

On Thu, 26 Jul 2007 13:00:01 +0300
Dotan Barak <dotanb at dev.mellanox.co.il> wrote:

> Joerg Zinke wrote:

> > ibv_modify_qp() fails with return value 22 when I try to open a new
> > CM connection under load (already ~3000 RDMA connections opened). I
> > tried to figure out what return value 22 means but could not find
> > it in the mthca kernel driver.
> >
> > Any hints? What does return value 22 mean?
> >   
> The value 22 is the ibv_modify_qp means that there was an invalid 
> parameter when calling to this verb.
> If you try to call to ibv_modify_qp without any load (only several
> QPs) do you still get this error?
> 

In short: no I do not get this error without load, because I start with
the no load situation and then more and more clients connecting until
~3000.

I have a simple CM server which accepts RDMA connections from
thousands clients. The code is based on the example/ stuff, with
the same simple handler functions to do the REQ/REP/RTU.
Everything is working fine, they all connect in the same manner (same
handler functions) until the point where ~3000 clients are connected and
the request handler fails on the server to modify the QP for the Reply
with return value 22.

With load i meant a lot of connections not the data transfer, 
because not all clients sending the whole time data through the RDMA
connections most of the time only one or two of the clients
sending... little integer pieces - so not much load on the lines.

Cheers 

Joerg