[ofa-general] RDS flow control

[Ralph Campbell]

On Tue, 2008-05-20 at 23:13 +0200, Olaf Kirch wrote:
> On Tuesday 20 May 2008 22:45:22 Jon Mason wrote:
> > Works well on my setup.
> 
> Good to hear!
> 
> > With proper flow control, there should no longer be a need for rnr_retry (as there
> > should always be a posted recv buffer waiting for the incoming data).  I did a
> > quick test and removed it and everything seemed to be happy on my rds-stress run.
> 
> I would like to make the setting of the RNR retry/timeout conditional on
> whether both ends of the connection support flow control or not - we need
> to think of rolling upgrades of a cluster, so mixed environments just have
> to work. Unfortunately, the RNR retry count is set prior to establishing
> the connection, before we even know whether the remote is capable of doing
> flow control.
> 
> Is there a way of changing the RNR retry count back to 0 after establishing
> the connection?

You can use ib_modify_qp() to set the QP state to IB_QPS_SQD (drain),
modify the IB_QP_RNR_RETRY parameter, and modify the QP back to
IB_QPS_RTS. It seems to me that modify QP could allow a RTS to RTS
transition and set the IB_QP_RNR_RETRY count but the qp_state_table[]
doesn't seem to indicate that is valid.