[openib-general] Solaris IPoIB MTU with OpenSM
Hal Rosenstock
halr at voltaire.com
Thu Feb 17 13:12:34 PST 2005
Hi Nitin,
On Wed, 2005-02-16 at 17:33, Nitin Hande wrote:
> On Wed, 2005-02-16 at 13:26, Hal Rosenstock wrote:
> > On Wed, 2005-02-16 at 16:08, Nitin Hande wrote:
> > > Hal,
> [snip..]
> > >
> > >
> > > Here is the trace of 256 sized MTU:
> > >
> > > Outgoing MAD:
> > > BaseVersion: 0x1
> > > MgmtClass: 0x3 - SubnAdm
> > > ClassVersion: 0x2
> > > R_Method: 0x12 - SubnAdmGetTable()
> > > Status: 0x0 - NO_ERROR
> > > ClassSpecific: 0x0
> > > TransactionID: 0x97651d100000096
> > > AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> > >
> > > 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> > > 0: 01 03 02 12 00 00 00 00 09 76 51 d1 00 00 00 96 .........vQ.....
> > > 10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 .8..............
> > > 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 30: 00 00 00 00 00 00 80 81 ff 12 40 1b ff ff 00 00 .......... at .....
> > > 40: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 ................
> > > 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 60: ff ff 00 00 00 00 00 00 20 00 00 00 00 00 00 00 ........ .......
> > > 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > >
> > > Incoming MAD:
> > > BaseVersion: 0x1
> > > MgmtClass: 0x3 - SubnAdm
> > > ClassVersion: 0x2
> > > R_Method: 0x92 -
> > > Status: 0x0 - NO_ERROR
> > > ClassSpecific: 0x0
> > > TransactionID: 0x97651d100000096
> > > AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> > >
> > > 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> > > 0: 01 03 02 92 00 00 00 00 09 76 51 d1 00 00 00 96 .........vQ.....
> > > 10: 00 38 00 00 ff ff ff ff 01 01 77 00 00 00 00 01 .8........w.....
> > > 20: 00 00 00 4c 00 00 00 00 00 00 00 00 00 07 00 00 ...L............
> > > 30: 00 00 00 00 00 00 80 81 ff 12 40 1b ff ff 00 00 .......... at .....
> > > 40: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 ................
> > > 50: 00 00 00 00 00 00 00 00 00 00 0b 1b c0 00 01 00 ................
> > > 60: ff ff 03 12 00 00 00 00 20 00 00 00 00 00 00 00 ........ .......
> > > 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > >
> > > And on other occassions where OpenSM reports the 2048 sized MTU:
> > >
> > > Outgoing MAD:
> > > BaseVersion: 0x1
> > > MgmtClass: 0x3 - SubnAdm
> > > ClassVersion: 0x2
> > > R_Method: 0x12 - SubnAdmGetTable()
> > > Status: 0x0 - NO_ERROR
> > > ClassSpecific: 0x0
> > > TransactionID: 0x97651d10000009a
> > > AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> > >
> > > 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> > > 0: 01 03 02 12 00 00 00 00 09 76 51 d1 00 00 00 9a .........vQ.....
> > > 10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 .8..............
> > > 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 30: 00 00 00 00 00 00 80 81 ff 12 40 1b ff ff 00 00 .......... at .....
> > > 40: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 ................
> > > 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 60: ff ff 00 00 00 00 00 00 20 00 00 00 00 00 00 00 ........ .......
> > > 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > >
> > > Incoming MAD:
> > > BaseVersion: 0x1
> > > MgmtClass: 0x3 - SubnAdm
> > > ClassVersion: 0x2
> > > R_Method: 0x92 -
> > > Status: 0x0 - NO_ERROR
> > > ClassSpecific: 0x0
> > > TransactionID: 0x97651d10000009a
> > > AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> > >
> > > 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> > > 0: 01 03 02 92 00 00 00 00 09 76 51 d1 00 00 00 9a .........vQ.....
> > > 10: 00 38 00 00 ff ff ff ff 01 01 77 00 00 00 00 01 .8........w.....
> > > 20: 00 00 00 4c 00 00 00 00 00 00 00 00 00 07 00 00 ...L............
> > > 30: 00 00 00 00 00 00 80 81 ff 12 40 1b ff ff 00 00 .......... at .....
> > > 40: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 ................
> > > 50: 00 00 00 00 00 00 00 00 00 00 0b 1b c0 00 04 00 ................
> > > 60: ff ff 03 12 00 00 00 00 20 00 00 00 00 00 00 00 ........ .......
> > > 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > > [output formatted manually]
> >
> > These both have the exactly selector issue which I fixed but I think you
> > haven't picked that up out of the tree. Is that a problem for Solaris or
> > does it ignore this in the response ?
> >
> > I have a theory for how the different MTUs (0 (256) and 4 (2048)) occur
> > but need a little time to validate it.
> So I patched the OpenSM with your changes and re-ran the same
> experiment. There was no difference in terms of response for
> MCMemberRecord issued by ibd driver:
>
> Here is just the response part:
>
> Incoming MAD:
> BaseVersion: 0x1
> MgmtClass: 0x3 - SubnAdm
> ClassVersion: 0x2
> R_Method: 0x92 -
> Status: 0x0 - NO_ERROR
> ClassSpecific: 0x0
> TransactionID: 0x97651d1000000a3
> AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
>
> 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> 0: 01 03 02 92 00 00 00 00 09 76 51 d1 00 00 00 a3 .........vQ.....
> 10: 00 38 00 00 ff ff ff ff 01 01 77 00 00 00 00 01 .8........w.....
> 20: 00 00 00 4c 00 00 00 00 00 00 00 00 00 07 00 00 ...L............
> 30: 00 00 00 00 00 00 80 81 ff 12 40 1b ff ff 00 00 .......... at .....
> 40: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 ................
> 50: 00 00 00 00 00 00 00 00 00 00 0b 1b c0 00 01 00 ................
> 60: ff ff 03 12 00 00 00 00 20 00 00 00 00 00 00 00 ........ .......
> 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>
> What did change though is the Selector in the McMemberRecord response
> when ipoib driver further tries to join this broadcast group and issues
> another MCMemberRecord.
>
>
>
> Outgoing MAD:
> BaseVersion: 0x1
> MgmtClass: 0x3 - SubnAdm
> ClassVersion: 0x2
> R_Method: 0x2 - SubnAdmSet()
> Status: 0x0 - NO_ERROR
> ClassSpecific: 0x0
> TransactionID: 0x97651d1000000a4
> AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
>
> 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> 0: 01 03 02 02 00 00 00 00 09 76 51 d1 00 00 00 a4 .........vQ.....
> 10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 .8..............
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 30: 00 00 00 00 00 01 b0 c7 ff 12 40 1b ff ff 00 00 .......... at .....
> 40: 00 00 00 00 ff ff ff ff fe 80 00 00 00 00 00 00 ................
> 50: 00 02 c9 01 09 76 51 d1 00 00 0b 1b 00 00 00 00 .....vQ.........
> 60: ff ff 00 00 00 00 00 00 21 00 00 00 00 00 00 00 ........!.......
> 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> Incoming MAD:
> BaseVersion: 0x1
> MgmtClass: 0x3 - SubnAdm
> ClassVersion: 0x2
> R_Method: 0x81 -
> Status: 0x0 - NO_ERROR
> ClassSpecific: 0x0
> TransactionID: 0x97651d1000000a4
> AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
>
> 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> 0: 01 03 02 81 00 00 00 00 09 76 51 d1 00 00 00 a4 .........vQ.....
> 10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 .8..............
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 07 00 00 ................
> 30: 00 00 00 00 00 01 b0 c7 ff 12 40 1b ff ff 00 00 .......... at .....
> 40: 00 00 00 00 ff ff ff ff fe 80 00 00 00 00 00 00 ................
> 50: 00 02 c9 01 09 76 51 d1 00 00 0b 1b c0 00 84 00 .....vQ.........
> 60: ff ff 83 92 00 00 00 00 21 00 00 00 00 00 00 00 ........!.......
> 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>
> Before the patch the selector was coming 04. Do you reply 84 seeing a
> particular component mask and otherwise 01 ??(I think not..)
I agree that OpenSM responds/should respond the same regardless of the
component mask in the request.
I was unaware of OpenSM responding with MTU of 01 until now. I have a
theory as to this. Any chance I can get the osm logs from a -V run of
the above ?
I also have a simple patch below to try which is just to test the
theory. This is off the latest version but should be easy to apply to
any version of osm_sa_mcmember_record.c.
This is separate from the support for PathRecords with multicast DGID
and/or DLID. I have the changes for this scoped out and should be able
to implement by early next week.
Thanks.
-- Hal
Index: osm_sa_mcmember_record.c
===================================================================
--- osm_sa_mcmember_record.c (revision 1821)
+++ osm_sa_mcmember_record.c (working copy)
@@ -1325,11 +1325,13 @@
/* copy qkey mlid tclass pkey sl_flow_hop mtu rate pkt_life
sl_flow_hop */
__copy_from_create_mc_rec(&mcmember_rec, &p_mgrp->mcmember_rec);
+#if 0
if(p_mgrp->well_known)
{
p_mgrp->mcmember_rec.mtu = mtu;
mcmember_rec.mtu = mtu;
}
+#endif
/* Release the lock as we don't need it. */
CL_PLOCK_RELEASE( p_rcv->p_lock );
More information about the general
mailing list