[openib-general] Re: Re: RFC: e2e credits

[Michael S. Tsirkin]

Quoting r. Sean Hefty <mshefty at ichips.intel.com>:
> Subject: Re: [openib-general] Re: Re: RFC: e2e credits
> 
> Michael S. Tsirkin wrote:
> >>The only way I can find for consumers to make use of this value is for 
> >>them to change their algorithms based on whether flow control is 
> >>supported or not.  But it's not clear to me how the consumer determines 
> >>what value to set flow control to.
> >
> >Oh, if you always post receive WQE before remote side posts send WQEs, then
> >you can disable flow control. SDP does this.
> 
> But as you're trying to address, there's no way to disable flow control.  
> I.e. this value doesn't get associated with the QP.  So, currently, the 
> only use that this value has is for the ULP to change its algorithm based 
> on whether or not it's set.

Right. Even if its associated with the QP, note how this is a
parameter of the remote peer's RQ, so you can't change it.

> My interpretation of the spec is that the value carried in the CM message 
> applies to the HCA as a whole, such that all REQs originating from the same 
> HCA should have the same value.  But I don't see how to determine what that 
> value should be.

Right, but I don't see that it matters much. If you think per-QP is nicer
we can change the meaning a bit.

> >>>>9.7.7.2.4 
> >>>>
> >>>>Even a responder which does generate end-to-end credits may choose
> >>>>to send the 'invalid' code in the AETH.
> >
> >
> >So an application can't really depend on the e2e credits for correctness -
> >its just an optimization hint.
> 
> Okay - I'm confused now.  Why have this value at all?  How does the 
> statement above match up with C9-150.2.1: For QPs that are not associated 
> with an SRQ, each HCA receive queue shall generate end-to-end flow control 
> credits?

Donnu.
All this is a bit theoretical since all ULPs today always post
receive WQEs before remote side posts sends.
So they all can just safely disable this bit and avoid the chance
of getting into limited state.

> >Anyway, sine ULPs don't seem to need it, another approach would be an 
> >option to
> >disable these flow controls globally, and probably add a module option to 
> >enable
> >them back just in case.  That's much simpler, isn't it?
> >
> >What do you think?
> 
> Examining the spec, it looks like flow control is defined as a per HCA 
> feature. Do we need an ib_device_cap_flag to indicate if flow control is 
> enabled or not on the device?

That could be fine, too.

-- 
Michael S. Tsirkin
Staff Engineer, Mellanox Technologies