[openib-general] tavor quirks etc (opensm compliance etc)

[Michael S. Tsirkin]

I'm really going off in a hurry, but for now:

> Quoting r. Or Gerlitz <ogerlitz at voltaire.com>:
> Subject: Re: tavor quirks etc (opensm compliance etc)
> 
> Michael S. Tsirkin wrote:
> >> 1) there are three patches
> > 
> > Not really, cma_tavor_quirk.patch is the *only* relevant tavor patch.
> > It is not 100% but the only work around for proprietary SMs.
> > Fixing the SA is a full solution.  We (Mellanox) will work with SA vendors to
> > get this addressed.  But of course this takes time.
> 
> cma_tavor_quirk.patch matches the patch you have applied to the opensm, 
>   so there are two patches at least, one at the stack and one at the 
> opensm. I don't think you can assume that all SA vendors would apply the 
> opensm approach and hence running with them the fixed cma tavor quirk as 
> which you have suggested today is useless with them (Specifically before 
> they even consider to apply it... so if someone runs OFED X.Y they would 
> not get 1K mtu with Tavor)

See below. With (fixed) cma_tavor_quirk.patch we are asking the SA
to give us a path with 1/2K MTU. If such path exists, SA should give it to us,
if it does not exist we should not try using it.

> > cma-tavor-quirk in OFED 1.1 is broken but not by design -
> > the patch I posted recently fixes the bug and should work with any compliant SM.
> > I did not look at the opensm code specifically, but the
> > "15.2.5.16 PATHRECORD" is quite explicit in its requirements:
> > 
> > MtuSelector 2 432 In a query request:
> >                      3-largest MTU available
> >                   If MTU is specified (i.e., the ComponentMask bit for
> >                   MTU is 1):
> >                      0-greater than MTU specified
> >                      1-less than MTU specified
> >                      2-exactly the MTU specified
> > 
> > So if e.g. opensm does not comply (e.g. it is not returning a path where one exists)
> > we should simply fix it. If there are other broken SMs, we can look at how they
> > are broken and how to solve this.
> 
> The SM team here don't think our SM is broken b/c it does not return 1K 
> path mtu where the minimal mtu as reported in the port info along the 
> path is 2k, and as i told you so does opensm without the quirk

Doesn't make sense to me, and I don't understand how you interpret MtuSelector.
Does the port really report minimal MTU 2K?
So how does lower MTU work at all then?
Are you saying some HCA/switch has a broken SMA?
Eitan?

> > 
> >> 3) the ipoib-selector patch (below) in a way assumes the open-sm quirk
> >> and hence it was not pushed upstream, and vise-versa an upstream ipoib
> >> code is broken with the open-sm running with the quirk!
> > 
> > All this is incorrect.  ipoib-selector is completely irrelevant to the MTU
> > issue - its a strict compliance fix for IPoIB. IPoIB also works fine without
> > this patch (with or without tavor quirk activated). It does not depend on any
> > specific SM. It is not upstream because of style issues only and due to my lack
> > of time to fix it. 
> 
> this reminds me that there is a need to do OFED 1.1 wrapup in the sense 
> we have to see which patches from the kernel_patches/fixes directory 
> were ***not*** pushed upstream to 2.6.19-rcX nor 2.6.20-rc1 and then 
> conduct some sort of discussion on each to decide what to do with it for 
> OFED 1.2

Most were pushed, a couple are outstanding. It's on my TODO,
but if you want to start working on it go ahead.

> > IB/ipoib: user appropriate mtu selector for path queries
> > 
> > IPoIB must set mtu selector in path record query according to dev->mtu:
> > if we wildcard it, SM can select a path with lower MTU.
> > This breaks IPoIB on networks with SM Tavor quirk activates.
> 
> mmm, re-reading the open sm code, i think you are right that the 
> ipoib-selector patch is independent of the open SM tavor quirk, but than 
> i don't understand what you were trying to say in the above two lines of 
> the change log, what can break the SM tavor quirk???
> 
> Or.

The change log is wrong there.


-- 
MST