[openib-general] Re: Re: Re: [PATCH] change Mellanox SDP workaround toa moduleparameter

[Sean Hefty]

Michael S. Tsirkin wrote:
>>>Okay, but then we need to implement the REJ/retry sequence.
>>>I think it affects at least CMA and maybe CM.
>>
>>I believe that this sort of retry need to be initiated by the ULP.
> 
> Okay, but we still need the CM to generate the approriate REJ with code 26,
> right?
> 
>>The CMA needs some additional work to permit re-using an rdma_cm_id after a 
>>connection has been rejected, so that the user or CMA can modify the MTU 
>>before trying to re-establish a connection.
> 
> Given that the issue only affects some hardware, it would be nice to somehow
> hide this from ULPs, otherwise it seems unlikely that they will get it right.

Are you wanting _all_ connections to this hardware to change the MTU?  I can see 
how this would be useful.  I was assuming that this was an SDP only related issue.

I'm not sure where we want this sort of policy.  I'm reluctant to mask this sort 
of connection change completely in either the IB CM or CMA.  We may still be 
able to locate the implementation there, but there should be someway for the 
user to override the settings.

Since this is a hardware specific problem, can the driver deal with this?  All 
received MADs are given to the driver before being processed.  Can the driver 
intercept the REQ, consume it, and issue a REJ?  This might be able to deal with 
the problem on the receive side.  The sender may also want to adjust the MTU 
based on local hardware, but I'm not sure where's the best place to handle this.

- Sean