[openib-general] failure using second hca via udapl

[mark kowalski]

I've been writing some code using udapl and recently added a second
hca to my machine.  Both hca's are mellanox cards:
0000:02:03.0 PCI bridge: Mellanox Technology MT23108 InfiniHost HCA
bridge (rev a1)
0000:03:00.0 InfiniBand: Mellanox Technology MT23108 InfiniHost HCA (rev a1)
0000:04:04.0 PCI bridge: Mellanox Technology MT23108 InfiniHost HCA
bridge (rev a0)
0000:05:00.0 InfiniBand: Mellanox Technology MT23108 InfiniHost HCA (rev a0)

both cards are recognized and seem to initialize fine:
Mar 15 17:40:37 kernel: Mellanox Tavor Device Driver is creating
device "InfiniHost0" (bus=03, devfn=00)
Mar 15 17:40:37 kernel: Mellanox Tavor Device Driver is creating
device "InfiniHost1" (bus=05, devfn=00)

the problem is when I try to access ports on the second hca I get this failure:
EVAPI_k_get_qp_hndl returns -244 (Invalid HCA Handle.) tsIbUCmAccept
failed: -5
I noticed this comment in the tsIbUCmAccept routine:
/* FIXME: Don't hardcode HCA handle for EVAPI_k_get_qp_hndl and
_tsIbUQpRegister */
after it set the qp_handle variable  to 0.  I modified the code to
pass the hca_handle that is input to tsIbUCmAccept function
(VAPI_hca_hndl_t                hca_handle)  in on the call to
EVAPI_k_get_qp_hndl instead of the qp_handle variable.  (i put some
print statements in the hca initialization code and the hca_handle for
the first hca was 0 and the hca_handle for the second hca was 1 so
this seemed like a reasonable thing to do since the hca_handle is an
index into the hca_tbl).

Anyway I get pass the EVAPI_k_get_qp_hndl failure but instead I get
this failure:

kernel: [KERNEL_IB][tsIbCmUserAccept][/var/tmp/IBGD/lib/modules/2.6.4-52-smp/build/drivers/infiniband/core/useraccess_cm.c:877]EVAPI_k_set_destroy_qp_cbk,
return code = -251 (Resource is busy)

in routine tsIbCmUserAccept  in the device driver code during the call
to this routine:
EVAPI_k_set_destroy_qp_cbk.  

I've gone through the initialization code and it seems that everything
that is done for the first hca is done for the second so it would seem
that once I pased in the correct hca_tbl index everything should work,
but it doesn't.
    Anyone have 2 hca's working via udapl out there?
Thanks,
Mark Kowalski