[ofa-general] Opensm compatibility with rate=1

[Greg Kurtzer]

On Jan 24, 2008 10:15 AM, Sasha Khapyorsky <sashak at voltaire.com> wrote:
> Hi Greg,
>
>
> On 15:18 Wed 23 Jan     , Greg Kurtzer wrote:
> >
> > We recently updated OFED (among other things) on one of our IB test
> > beds that use older cards. Something broke recently with an error in
> > dmesg like:
> >
> > kernel: ib0: multicast join failed for
> > ff12:401b:ffff:0000:0000:0000:ffff:ffff, status -22
> >
> > We used to fix this by defining our partition to be:
> >
> > Default=0x7fff,ipoib,rate=1:ALL=full;
> >
> > But this no longer seems to work.
> >
> > In the opensm source code I see the following:
> >
> >
> > /* following v1 ver1.2 p901 */
> > #define IB_PATH_RECORD_RATE_2_5_GBS             2
> > #define IB_PATH_RECORD_RATE_10_GBS              3
> > #define IB_PATH_RECORD_RATE_30_GBS              4
> > #define IB_PATH_RECORD_RATE_5_GBS               5
> > #define IB_PATH_RECORD_RATE_20_GBS              6
> > #define IB_PATH_RECORD_RATE_40_GBS              7
> > #define IB_PATH_RECORD_RATE_60_GBS              8
> > #define IB_PATH_RECORD_RATE_80_GBS              9
> > #define IB_PATH_RECORD_RATE_120_GBS             10
> >
> > #define IB_MIN_RATE    IB_PATH_RECORD_RATE_2_5_GBS
> > #define IB_MAX_RATE    IB_PATH_RECORD_RATE_120_GBS
> >
> > Which forces the lowest possible rate to be 2 which doesn't work with
> > our test bed. By kludging IB_MIN_RATE to be set to 1, things seem to
> > be working but chances are supporting only rates >= 2 was done on
> > purpose. Is there a better workaround or solution to this, or a way of
> > continuing support for rate=1?
>
> What is the purpose of rate=1 in your setup? According to IBA spec the
> value '1' for rate is "reserved".

Hrmm, that is interesting. The command "ibstatus" shows that the rate
is 2.5 Gb/sec (1X) yet "ibstat" shows the rate being 2. The opensm -V
logs shows that it was setting the rate to 2 as well (which didn't
work). Even by me forcing the IB_MIN_RATE to 1 still yields the same
output, but opensm just works now with no more error messages.

I can't tell you anything more then that except that earlier versions
of opensm worked with a rate of 1.

Thanks!
Greg
-- 
Greg Kurtzer
http://www.runlevelzero.net/