[ewg] rdma_accept and non-rdma QPs

[Dries Kimpe]

Hi,

I've been having some issues in using RDMA CM with QPs created and owned
by the applications, i.e. created with ibv_create_qp as opposed to
rdma_create_qp.

The man pages for both rdma_accept and rdma_connect hint that this is
possible.

In my application, whem I'm using rdma_create_qp, everything works.
If I change to using ibv_create_qp (but everything else the same:
qp_init_attr, PD, ... -- only change is rdma_create_qp -> ibv_create_qp)
the call to rdma_accept fails with 'invalid argument'.

[I do correctly specify the QP number in the accept parameters and
connect parameters in both cases. The documentation states that rdmacm
ignores qp_num in case rdma_create_qp was called]

Some more details:
- RC connection
- Ofed version doesn't seem to matter (tried up to 2.1).
- Using my own protection domain (not created by RDMA CM) but derived
  from the same context to which the ID is bound.
- Using outside (i.e. not RDMA CM created) completion queue

Things I've verified/tried:

- verify context for RDMA cm id and QP created by ibv_create_qp is the
  same.
- verify protection domains belong to the same context.
- transition QP to INIT on passive side before calling rdma_accept.
  (I checked the rdma cm source code and noticed that rdma_create_qp
  also does this)
- Checked the kernel code to see where an invaliad argument parameter
  could be returned but nothing stood out.
- Verified sequence of calls on both active and passive side
  as described in rdma_cm manpage.

I wasn't able to find any example code using rdma cm but not
rdma_create_qp.
It would be very helpful if somebody could list the differences or point
me to some code.

   Thanks,
   Dries