[ofa-general] RDS flow control

[Richard Frank]

Steve Wise wrote:
> Olaf Kirch wrote:
>> On Monday 12 May 2008 18:57:38 Jon Mason wrote:
>>  
>>> As part of my effort to get RDS working for iWARP, I will be working 
>>> on the RDS flow control.  Flow control is needed for iWARP due to 
>>> the fact that iWARP connections terminate if there is no posted recv 
>>> for an incoming packet.  IB connections do not have this limitation 
>>> if setup in a certain way.  In its current implementation, RDS sets 
>>> the connection attribute rnr_retry to 7.  This causes IB to 
>>> retransmit until there is a posted recv buffer.     
>>
>> I think for the initial implementation, it is fine for iWARP to just
>> fail the connect when that happens, and re-establish the connection.
>>
>> If you use reasonable defaults for the send and recv queues, receiver
>> overruns should be relatively rare.
>>
>> Once everything else works, let's revisit the flow control part.
>>
>>   
> I _think_ you'll hit this quickly with one-way flows.  Send 
> completions for iWARP only mean the user's buffer can be reused.  Not 
> that its placed at the remote peer or in the remote user's buffer.
>
Let's see what happens - anyway - this could be solved in an IWARP 
extension to RDS  - right ?
> But perhaps I'm wrong.  Jon, maybe you should try to hit this with IB 
> and rnr_retry == 0 using the rds perf tools?
> Also "the everything else" part depends on remove fmr usage.  I'm 
> working on the new RDMA memory verbs allowing fast registration of 
> physical memory via a send WR.  To support iWARP we need to remove the 
> fmr usage from RDS.   The idea was to replace fmrs with the new 
> fastreg verbs.   Thoughts?
>
What does "fast" imply here - how does this compare to the performance 
of FMRs ?

Why would not push memory window creation into the RDS transport 
specific implementations ?

Changing the API may be OK - if we retain the performance we have with IB.

> Stay tuned for the new verbs API RFC...
>
> Steve.
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general