[openib-general] Re: Re: RFC: e2e credits

[Michael S. Tsirkin]

Quoting r. Sean Hefty <mshefty at ichips.intel.com>:
> Subject: Re: [openib-general] Re: Re: RFC: e2e credits
> 
> Michael S. Tsirkin wrote:
> >I think that the CM flow control parameter affects the requestor.
> >This option is for responder.
> 
> The CM flow control parameter indicates if the local HCA implements flow 
> control or always advertises invalid credits.  It's carried in both the REQ 
> and REP messages, so we end up knowing the value used by both sides.  (See 
> 12.7.26)
>
> The only way I can find for consumers to make use of this value is for them 
> to change their algorithms based on whether flow control is supported or 
> not.  But it's not clear to me how the consumer determines what value to 
> set flow control to.

Oh, if you always post receive WQE before remote side posts send WQEs, then
you can disable flow control. SDP does this.

But:

>>> 9.7.7.2.4 
>>>
>>> Even a responder which does generate end-to-end credits may choose
>>> to send the 'invalid' code in the AETH.

So an application can't really depend on the e2e credits for correctness -
its just an optimization hint.

> I think that we need to relate anything that we do with flow control in 
> with the value exchanged as part of the IB CM protocol.  (This value is 
> also exposed through the RDMA CM as well.)  From an architectural view, 
> this may require changing the meaning of the value carried in the IB CM 
> messages from 'signifying whether the local CA actually implements flow 
> control', to 'signifying whether the local QP will implement flow control'.

I guess its fine to do it this way - its just an optimization hint to the
application.

Anyway, sine ULPs don't seem to need it, another approach would be an option to
disable these flow controls globally, and probably add a module option to enable
them back just in case.  That's much simpler, isn't it?

What do you think?

-- 
Michael S. Tsirkin
Staff Engineer, Mellanox Technologies