[ofa-general] ibv_post_send returns -1

[Dotan Barak]

On Sat, Dec 27, 2008 at 10:17 PM, Bharath Ramesh <bramesh at vt.edu> wrote:
> Thanks for the reply Dotan, I am posting what I am doing currently, probably
> I am missing something which you could point out to me.
>
> 1) I don't have completion notifications or completion channel associated
> with my send CQ.
> 2) I only have RC QPs.
> 3) Every few hundred messages (256 messages to be precise) for my small
> messages (64 bytes long) for which I use IBV_WR_SEND with IBV_SEND_INLINE I
> set the IBV_SEND_SIGNALED flag. Once I set the flag I poll my send CQ for
> completion messages.
> 4) All my RDMA buffers for large messages are all registered, and I use
> IBV_SEND_SIGNALED flag for the WR and poll the send CQ for completion
> immediately after ibv_post_send.
>
> The only time when I don't poll my send CQ for completion for ibv_post_send
> is for my small messages. I presume that polling the send CQ once every 256
> messages for these should be enough to free up the WR from the SQ for the
> previously posted WR. Am I wrong in assuming this, should each and every
> ibv_post_send should be associated with poll to the send CQ?
>
No, you are right. You can poll the CQ once in a while and not after
every post, but you have to make sure that the total number
of outstanding WR in the Send Queue is less that the value that the QP
was created with.

Dotan