[openib-general] Solaris IPoIB MTU with OpenSM

Nitin Hande Nitin.Hande at Sun.COM
Wed Feb 16 14:33:39 PST 2005 

On Wed, 2005-02-16 at 13:26, Hal Rosenstock wrote:
> On Wed, 2005-02-16 at 16:08, Nitin Hande wrote:
> > Hal,
[snip..]
> > 
> > 
> > Here is the trace of 256 sized MTU:
> > 
> > Outgoing MAD:
> >         BaseVersion: 0x1
> >         MgmtClass: 0x3 - SubnAdm
> >         ClassVersion: 0x2
> >         R_Method: 0x12 - SubnAdmGetTable()
> >         Status: 0x0 - NO_ERROR
> >         ClassSpecific: 0x0
> >         TransactionID: 0x97651d100000096
> >         AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> > 
> >     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
> >  0: 01 03 02 12 00 00 00 00 09 76 51 d1 00 00 00 96  .........vQ.....
> > 10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  .8..............
> > 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 30: 00 00 00 00 00 00 80 81 ff 12 40 1b ff ff 00 00  .......... at .....
> > 40: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  ................
> > 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 60: ff ff 00 00 00 00 00 00 20 00 00 00 00 00 00 00  ........ .......
> > 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 
> > Incoming MAD:
> >         BaseVersion: 0x1
> >         MgmtClass: 0x3 - SubnAdm
> >         ClassVersion: 0x2
> >         R_Method: 0x92 -
> >         Status: 0x0 - NO_ERROR
> >         ClassSpecific: 0x0
> >         TransactionID: 0x97651d100000096
> >         AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> > 
> >     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
> >  0: 01 03 02 92 00 00 00 00 09 76 51 d1 00 00 00 96  .........vQ.....
> > 10: 00 38 00 00 ff ff ff ff 01 01 77 00 00 00 00 01  .8........w.....
> > 20: 00 00 00 4c 00 00 00 00 00 00 00 00 00 07 00 00  ...L............
> > 30: 00 00 00 00 00 00 80 81 ff 12 40 1b ff ff 00 00  .......... at .....
> > 40: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  ................
> > 50: 00 00 00 00 00 00 00 00 00 00 0b 1b c0 00 01 00  ................
> > 60: ff ff 03 12 00 00 00 00 20 00 00 00 00 00 00 00  ........ .......
> > 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 
> > And on other occassions where OpenSM reports the 2048 sized MTU:
> > 
> > Outgoing MAD:
> >         BaseVersion: 0x1
> >         MgmtClass: 0x3 - SubnAdm
> >         ClassVersion: 0x2
> >         R_Method: 0x12 - SubnAdmGetTable()
> >         Status: 0x0 - NO_ERROR
> >         ClassSpecific: 0x0
> >         TransactionID: 0x97651d10000009a
> >         AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> > 
> >     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
> >  0: 01 03 02 12 00 00 00 00 09 76 51 d1 00 00 00 9a  .........vQ.....
> > 10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  .8..............
> > 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 30: 00 00 00 00 00 00 80 81 ff 12 40 1b ff ff 00 00  .......... at .....
> > 40: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  ................
> > 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 60: ff ff 00 00 00 00 00 00 20 00 00 00 00 00 00 00  ........ .......
> > 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 
> > Incoming MAD:
> >         BaseVersion: 0x1
> >         MgmtClass: 0x3 - SubnAdm
> >         ClassVersion: 0x2
> >         R_Method: 0x92 -
> >         Status: 0x0 - NO_ERROR
> >         ClassSpecific: 0x0
> >         TransactionID: 0x97651d10000009a
> >         AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> > 
> >     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
> >  0: 01 03 02 92 00 00 00 00 09 76 51 d1 00 00 00 9a  .........vQ.....
> > 10: 00 38 00 00 ff ff ff ff 01 01 77 00 00 00 00 01  .8........w.....
> > 20: 00 00 00 4c 00 00 00 00 00 00 00 00 00 07 00 00  ...L............
> > 30: 00 00 00 00 00 00 80 81 ff 12 40 1b ff ff 00 00  .......... at .....
> > 40: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  ................
> > 50: 00 00 00 00 00 00 00 00 00 00 0b 1b c0 00 04 00  ................
> > 60: ff ff 03 12 00 00 00 00 20 00 00 00 00 00 00 00  ........ .......
> > 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> > [output formatted manually]
> 
> These both have the exactly selector issue which I fixed but I think you
> haven't picked that up out of the tree. Is that a problem for Solaris or
> does it ignore this in the response ?
> 
> I have a theory for how the different MTUs (0 (256) and 4 (2048)) occur
> but need a little time to validate it.
So I patched the OpenSM with your changes and re-ran the same
experiment. There was no difference in terms of response for
MCMemberRecord issued by ibd driver: 

Here is just the response part:

Incoming MAD:
        BaseVersion: 0x1
        MgmtClass: 0x3 - SubnAdm
        ClassVersion: 0x2
        R_Method: 0x92 -
        Status: 0x0 - NO_ERROR
        ClassSpecific: 0x0
        TransactionID: 0x97651d1000000a3
        AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID

 0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
 0: 01 03 02 92 00 00 00 00 09 76 51 d1 00 00 00 a3  .........vQ.....
10: 00 38 00 00 ff ff ff ff 01 01 77 00 00 00 00 01  .8........w.....
20: 00 00 00 4c 00 00 00 00 00 00 00 00 00 07 00 00  ...L............
30: 00 00 00 00 00 00 80 81 ff 12 40 1b ff ff 00 00  .......... at .....
40: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  ................
50: 00 00 00 00 00 00 00 00 00 00 0b 1b c0 00 01 00  ................
60: ff ff 03 12 00 00 00 00 20 00 00 00 00 00 00 00  ........ .......
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................

What did change though is the Selector in the McMemberRecord response
when ipoib driver further tries to join this broadcast group and issues
another MCMemberRecord. 



Outgoing MAD:
        BaseVersion: 0x1
        MgmtClass: 0x3 - SubnAdm
        ClassVersion: 0x2
        R_Method: 0x2 - SubnAdmSet()
        Status: 0x0 - NO_ERROR
        ClassSpecific: 0x0
        TransactionID: 0x97651d1000000a4
        AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID

    0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
 0: 01 03 02 02 00 00 00 00 09 76 51 d1 00 00 00 a4  .........vQ.....
10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  .8..............
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
30: 00 00 00 00 00 01 b0 c7 ff 12 40 1b ff ff 00 00  .......... at .....
40: 00 00 00 00 ff ff ff ff fe 80 00 00 00 00 00 00  ................
50: 00 02 c9 01 09 76 51 d1 00 00 0b 1b 00 00 00 00  .....vQ.........
60: ff ff 00 00 00 00 00 00 21 00 00 00 00 00 00 00  ........!.......
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
Incoming MAD:
        BaseVersion: 0x1
        MgmtClass: 0x3 - SubnAdm
        ClassVersion: 0x2
        R_Method: 0x81 -
        Status: 0x0 - NO_ERROR
        ClassSpecific: 0x0
        TransactionID: 0x97651d1000000a4
        AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID

   0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
 0: 01 03 02 81 00 00 00 00 09 76 51 d1 00 00 00 a4  .........vQ.....
10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  .8..............
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 07 00 00  ................
30: 00 00 00 00 00 01 b0 c7 ff 12 40 1b ff ff 00 00  .......... at .....
40: 00 00 00 00 ff ff ff ff fe 80 00 00 00 00 00 00  ................
50: 00 02 c9 01 09 76 51 d1 00 00 0b 1b c0 00 84 00  .....vQ.........
60: ff ff 83 92 00 00 00 00 21 00 00 00 00 00 00 00  ........!.......
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................

Before the patch the selector was coming 04. Do you reply 84 seeing a
particular component mask and otherwise 01 ??(I think not..) 


Thanks
Nitin





> 
> -- Hal
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general

More information about the general mailing list