[openib-general] QP creation failure

[Raleigh F Rinehart]

Hi,
I'm hoping someone here can help me diagnose a this problem.
I have a really simple test app that uses verbs and is failing to create 
a QP on one machine in particular.  On other machines the app works and 
behaves as expected without any problems.

The machine in question is  32bit dual CPU Intel system running FC4 and 
the released OFED 1.1 with a Mellanox PCI-X HCA (MT23108)
[root at localhost test]# uname -a
Linux localhost.localdomain 2.6.11-1.1369_FC4smp #1 SMP Thu Jun 2 
23:08:39 EDT 2005 i686 i686 i386 GNU/Linux
[root at localhost test]# cat /usr/local/ofed/BUILD_ID
OFED-1.1

openib-1.1 (REV=9905)
# User space
https://openib.org/svn/gen2/branches/1.1/src/userspace
Git:
ref: refs/heads/ofed_1_1
commit a083ec1174cb4b5a5052ef5de9a8175df82e864a

The code in question is pretty simple and as I've said works everywhere 
else I've tried it.

Errno is set to 22, and I've traced the problem to this point in the 
OFED stack, so I can see where it fails but still have no idea why:
It fails at line 578 in "src/userspace/libibverbs/src/cmd.c" the 
instruction is
'write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size'
cmd_fd looked valid (was 6), cmd looked to point to a valid structure,  
and cmd_size was 96.

This was called from line 533 of src/userspace/libmthca/src/verbs.c 
'ibv_cmd_create_qp(pd, &qp->ibv_qp, attr, &cmd.ibv_cmd, sizeof cmd, &resp, sizeof resp);'

Which was invoked by my code calling ibv_create_qp as seen below:

<snip>
 /* create the qpairs */
                        init_attr.send_cq = info->cq_hndl;
                        init_attr.recv_cq = info->cq_hndl;
                        init_attr.cap.max_send_wr  = info->oust_wr_sq; //8
                        init_attr.cap.max_recv_wr  = info->oust_wr_rq; //8
                        init_attr.cap.max_send_sge = info->sg_size_sq; //1
                        init_attr.cap.max_recv_sge = info->sg_size_rq; //1
                        init_attr.cap.max_inline_data = 1024;
                        init_attr.qp_type = IBV_QPT_RC;

                        if ((info->qp_hndl[CLIENT] = 
ibv_create_qp(info->pd_hndl, &init_attr)) == NULL) {
                               
                                info->failed = 1;
                                rc = ERR_INIT_HCA_FAILED;
                               
                        }
</snip>


Any ideas or pointers in the right direction would be greatly appreciated.


-raleigh
raleigh AT systemfabricworks.com