[ewg] not using IBV_SEND_SIGNALED

[Steve Wise]

On 11/29/2010 11:34 AM, Robert D. Russell wrote:
> Steve:
>
> Thanks, that fix works fine.
> Is the need to do this documented anywhere?
>

See section 8.2.1 of the iWARP Verbs draft at:

http://tools.ietf.org/html/draft-hilland-rddp-verbs-00#section-8.2.1.

The last big paragraph in 8.2.1 spells it all out.

There should be a similar requirement in the IBTA IB Verbs specification...

Steve.



> Thanks,
> Bob Russell
> rdr at iol.unh.edu
>
>
> On Mon, 29 Nov 2010, Steve Wise wrote:
>
>>
>> On 11/29/2010 10:31 AM, Robert D. Russell wrote:
>>> Hi:
>>>
>>> I have hit the following strange situation and would appreciate an
>>> explanation as to how to fix it.
>>>
>>> When calling ibv_create_cq() to create a queue pair, I set 
>>> sq_sig_all = 0.
>>> When calling ibv_post_send() I set send_flags in the work request to 0
>>> (so that IBV_SEND_SIGNALED is NOT set).
>>> When calling ibv_poll_cq() I do not get any IBV_WC_SEND completions.
>>> I do not create a completion channel, nor do I use ibv_req_notify_cq()
>>> or ibv_get_cq_event() or ibv_ack-cq_events().
>>> So far, so good, and this works for a few ibv_post_send()s.
>>>
>>> However, as soon as I call ibv_post_send() more than max_send_wr times,
>>> it fails on Mellanox IB cards, and Intel and Chelsio iWARP cards,
>>> but it works fine on QLogic IB cards.
>>> It appears that the send work completions ARE being generated 
>>> incorrectly
>>> (or at least some internal counter is being incremented incorrectly)
>>> even 'tho' they are correctly NOT being put into the completion queue.
>>> Is there some other way to clear this counter or to get this to work?
>>>
>>> Thanks,
>>> Bob Russell
>>> rdr at iol.unh.edu
>>>
>>
>> You must signal one WR for every SQ-depth worth of WRs posted.  IE: 
>> If your SQ depth is 16, you must signal at least one out of every 
>> 16.  This ensures proper flow control for HW resources.
>>
>> Steve.
>>
>>
>>