[ofa-general] ibverbs help

[Dotan Barak]

Kelly Burkhart wrote:
> Hello,
>
> I have written a simple program as my first foray into verbs
> programming.  The intent is to create a connection and do a single send.
> The logic (ripped primarily from rc_pingpong) is this:
>
> get device
> ibv_open_device
> ibv_alloc_pd
> allocate buffer
> ibv_reg_mr
> ibv_create_cq
> ibv_create_qp
> ibv_modify_qp  (set to INIT)
> ibv_query_port (exchange lid/qp_num with peer)
> ibv_modify_qp  (set to RTR, set remote qpnum, lid)
> ibv_modify_qp  (set to RTS)
>
> SERVER:
>   ibv_post_recv
>   ibv_poll_cq  (loop)
>
> CLIENT:
>   ibv_post_send
>   ibv_poll_cq  (loop)
>
> My assumption is that when I call ibv_post_send or ibv_post_recv, that I
> should be able to receive notification of the completion of that call by
> spinning on ibv_poll_cq until the work completion is available.
> However, my program spins on ibv_poll_cq indefinitely.  Is there an
> error in my understanding?  Or an error in my program (attached).
>
> To run:
>
> test_ibv_simple  (run as server)
> test_ibv_simple asdf  (any arg will cause client)
>   
I think that i found a bug in you code:
<start>
while( (ne = ibv_poll_cq( cq_, 1, &wc )) < 1 );
if (wc.status != IBV_WC_SUCCESS)
        throw std::runtime_error( "Channel::read -- ibv_poll_cq revealed 
failed recv" );
<end>

If there is a problem in the CQ and the return value is negative, you 
won't notice it ...


I'm quite sure that this is not the only issue in your code.

Dotan