[openib-general] Re: RFC: e2e credits

[Hal Rosenstock]

On Thu, 2006-03-23 at 12:34, Michael S. Tsirkin wrote:
> Quoting r. Hal Rosenstock <halr at voltaire.com>:
> > > > > > > > > >Sean, just to wrap it up, the API at the verbs layer will look
> > > > > > > > > >like the below, and then ULPs just put the value they want in
> > > > > > > > > >the CM and CM will pass it in to low level.
> 
> So this is our question, right?
> 
> 
> CM REQ and REP messages include the following field:
> 
> ---------------
> 12.7.26 END-TO-END FLOW CONTROL
> Signifies whether the local CA actually implements End-to-End Flow Control
> (1), or instead always advertises .invalid credits.(0). See section
> 9.7.7.2 End-to-End (Message Level) Flow Control for more detail.
> ---------------
> 
> Consider and implementation that advertises valid credits for
> connections, and always advertises invalid credits for other connections.
> This is compliant since the IB spec says (end-to-end (message level) flow
> control, Requester Behaviour):
> "Even a responder which does generate end-to-end credits may choose to send the
> 'invalid' code in the AETH"

I did some spec reading to find this and found the following which I
think makes the current requirement clear:

p.347 line 37 states "HCA receive queues must generate end-to-end
credits (except for QPs associated with a SRQ), but TCA receive queues
are not required to do so. This appears to be informative text. I first
found the following:

p. 348 has the compliances for this for both HCAs and TCAs:

C9-150.2.1: For QPs that are not associated with an SRQ, each HCA re-
ceive queue shall generate end-to-end flow control credits. If a QP is
associated with an SRQ, the HCA receive queue shall not generate end-to-
end flow control credits.

o9-95.2.1: Each TCA receive queue may generate end-to-end credits ex-
cept for QPs that are associated with an SRQ. If a TCA supports SRQ, the
TCA must not generate End-to-End Flow Control Credits for QPs associ-
ated with an SRQ.

C9-151: If a TCA's given receive queue generates End-to-End credits,
then the corresponding send queue shall receive and respond to those
credits. This is a requirement on each send queue of a CA.

The above informative text also references the CA requirements in
chapter 17 and on p. 1026 line 25 there is a row in the table for end to
end flow control for RC consistent with the above. p.1028 has the
compliances for this.

> Is it compliant for CM implementations to set/clear the End-to-End Flow Control
> field accordingly, taking it to mean
> 
> "whether the local CA actually implements End-to-End Flow Control
> (1), or instead always advertises 'invalid credits'(0)"
> *for the specific connection*

So IMO the intent of what was written is clear (on a per CA basis) and
this is a spec change which is OK to propose but needs a different
writeup.

-- Hal