[ewg] not using IBV_SEND_SIGNALED

[Robert D. Russell]

Steve:

Thanks, that fix works fine.
Is the need to do this documented anywhere?

Thanks,
Bob Russell
rdr at iol.unh.edu


On Mon, 29 Nov 2010, Steve Wise wrote:

>
> On 11/29/2010 10:31 AM, Robert D. Russell wrote:
>> Hi:
>> 
>> I have hit the following strange situation and would appreciate an
>> explanation as to how to fix it.
>> 
>> When calling ibv_create_cq() to create a queue pair, I set sq_sig_all = 0.
>> When calling ibv_post_send() I set send_flags in the work request to 0
>> (so that IBV_SEND_SIGNALED is NOT set).
>> When calling ibv_poll_cq() I do not get any IBV_WC_SEND completions.
>> I do not create a completion channel, nor do I use ibv_req_notify_cq()
>> or ibv_get_cq_event() or ibv_ack-cq_events().
>> So far, so good, and this works for a few ibv_post_send()s.
>> 
>> However, as soon as I call ibv_post_send() more than max_send_wr times,
>> it fails on Mellanox IB cards, and Intel and Chelsio iWARP cards,
>> but it works fine on QLogic IB cards.
>> It appears that the send work completions ARE being generated incorrectly
>> (or at least some internal counter is being incremented incorrectly)
>> even 'tho' they are correctly NOT being put into the completion queue.
>> Is there some other way to clear this counter or to get this to work?
>> 
>> Thanks,
>> Bob Russell
>> rdr at iol.unh.edu
>> 
>
> You must signal one WR for every SQ-depth worth of WRs posted.  IE: If your 
> SQ depth is 16, you must signal at least one out of every 16.  This ensures 
> proper flow control for HW resources.
>
> Steve.
>
>
>