[ofa-general] ibv_post_send returns -1

[Dotan Barak]

Bharath Ramesh wrote:
> I using OFED-1.2 for my research project. I am facing an issue where 
> when I try to post IBV_WR_RDMA_WRITE work request to the QP 
> ibv_post_send returns with -1. This I presume means lack of resources 
> to complete the operation. To give a brief summary of my communication 
> mechanism, I have two kind of messages control messages and large 
> messages. I post control messages with the normal IBV_WR_SEND with 
> send flags set to IBV_SEND_INLINE. Every few hundred such control 
> messages I set the send flag to IBV_SEND_INLINE | IBV_SEND_SIGNALED 
> and poll my send CQ. For large messages I use IBV_WR_RDMA_WRITE with 
> send flags set to IBV_SEND_SIGNALED. After posting any large message I 
> poll the send CQ for completion. I have sufficient buffers posted on 
> the receiver.
>
> The first thing I did when I encountered this issue is to write a 
> small test which just does only IBV_WR_RDMA_WRITE similar to my 
> application. This test works fine without having any issues. I am 
> wondering what could be potential issues that might be causing such a 
> scenario of unavailable resources. Any help in debugging this 
> situation would be appreciated. I am not subscribed to the list would 
> appreciate if I am copied on the replies.
Hi.


ibv_post_send may fail if:
1) too many WR are outstanding on the SQ (maybe you posted too many WR 
without polling for there completion)
2) bad WR was posted (for example: RDMA Write for UD QP).
3) maybe the size of the WR that you are trying to post as inline is too 
big (upon QP creation you can see this size in the sq_send_inline)


Dotan