[openib-general] QP creation failure

[dotanb at dev.mellanox.co.il]

Hi.


> Hi,
> I'm hoping someone here can help me diagnose a this problem.
> I have a really simple test app that uses verbs and is failing to create
> a QP on one machine in particular.  On other machines the app works and
> behaves as expected without any problems.
>
> The machine in question is  32bit dual CPU Intel system running FC4 and
> the released OFED 1.1 with a Mellanox PCI-X HCA (MT23108)
> [root at localhost test]# uname -a
> Linux localhost.localdomain 2.6.11-1.1369_FC4smp #1 SMP Thu Jun 2
> 23:08:39 EDT 2005 i686 i686 i386 GNU/Linux
> [root at localhost test]# cat /usr/local/ofed/BUILD_ID
> OFED-1.1
>
> openib-1.1 (REV=9905)
> # User space
> https://openib.org/svn/gen2/branches/1.1/src/userspace
> Git:
> ref: refs/heads/ofed_1_1
> commit a083ec1174cb4b5a5052ef5de9a8175df82e864a
>
> The code in question is pretty simple and as I've said works everywhere
> else I've tried it.
>
> Errno is set to 22, and I've traced the problem to this point in the
> OFED stack, so I can see where it fails but still have no idea why:
> It fails at line 578 in "src/userspace/libibverbs/src/cmd.c" the
> instruction is
> 'write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size'
> cmd_fd looked valid (was 6), cmd looked to point to a valid structure,
> and cmd_size was 96.
>
> This was called from line 533 of src/userspace/libmthca/src/verbs.c
> 'ibv_cmd_create_qp(pd, &qp->ibv_qp, attr, &cmd.ibv_cmd, sizeof cmd, &resp,
> sizeof resp);'
>
> Which was invoked by my code calling ibv_create_qp as seen below:
>
> <snip>
>  /* create the qpairs */
>                         init_attr.send_cq = info->cq_hndl;
>                         init_attr.recv_cq = info->cq_hndl;
>                         init_attr.cap.max_send_wr  = info->oust_wr_sq; //8
>                         init_attr.cap.max_recv_wr  = info->oust_wr_rq; //8
>                         init_attr.cap.max_send_sge = info->sg_size_sq; //1
>                         init_attr.cap.max_recv_sge = info->sg_size_rq; //1
>                         init_attr.cap.max_inline_data = 1024;
>                         init_attr.qp_type = IBV_QPT_RC;
>
>                         if ((info->qp_hndl[CLIENT] =
> ibv_create_qp(info->pd_hndl, &init_attr)) == NULL) {
>
>                                 info->failed = 1;
>                                 rc = ERR_INIT_HCA_FAILED;
>
>                         }
> </snip>
>
>
> Any ideas or pointers in the right direction would be greatly appreciated.

I think that the problem is the amount of inline data that you try to use.
I suggest that you put 0, create the QP and check the value that are being
returned from the QP creation and use it.

I believe that the maximum size that can be used in this attribute is ~ 420 .


Dotan