[ofw] IBV_WC_LOC_PROT_ERR

[Sean Hefty]

>When I do ibv_post_send() call (IBV_WR_SEND), I get
>'IBV_WC_LOC_PROT_ERR' error in WC status  in  the send CQ hander.
>Is this related to lkey ? I create mr for the send  buffer with
>WV_ACCESS_LOCAL_WRITE access.

Either the lkey is wrong, or the address/length falls outside of the registered
memory range.

>The first call to ibv_post_send() completes with IBV_WC_SUCCESS.
>The second call to  ibv_post_send() fails with the above error.  The
>lkey here is same as the first  post send and use different range in
>send buffer for which MR is obtained.
>The subsequent calls to ibv_post_send() completes with IBV_WC_WR_FLUSH_ERR.

Btw - this explains your 'missing' receive completion issue, since the send
never got there.  If you can post your send code, it can help us debug it.