[openib-general] Solaris IPoIB MTU with OpenSM
Hal Rosenstock
halr at voltaire.com
Thu Feb 24 05:42:23 PST 2005
Hi Nitin,
On Wed, 2005-02-23 at 17:19, Nitin Hande wrote:
> Hal,
>
> [comments below]
> On Wed, 2005-02-23 at 02:19, Hal Rosenstock wrote:
> > On Tue, 2005-02-22 at 22:56, Nitin Hande wrote:
> > > So I tried the latest patches and preliminarily things seem to be
> > > working fine.
> >
> > Yipee.
> [snip..]
> >
> > >
> > > So after this test above, I try to run snoop on the solaris interface
> > > and get the following error message from the layer below IPoIB:
> > >
> > > Feb 22 19:50:25 dongon.SFBay.Sun.COM ibd: [ID 517869 kern.info] NOTICE:
> > > ibd0: HCA GUID 0002c901097651d0 port 1 PKEY ffff Could not get list of
> > > IBA multicast groups
> > >
> > > My preliminary assumption is that OpenSm is not returning the list of
> > > multicast groups that the ibd interface has joined. I will look at the
> > > MAD's tomorrow and try to ascertain that.
> >
> > How does S10 request this ? Remember that if it is a GetTable and
> > doesn't fit in a single MAD, it will be broken now. If that is the case,
> > we will live with this until we have real RMPP.
> Below is an an example of a single GetTable request and response between
> Solaris and OpenSM. OpenSM is not reporting the MCgroups in case of a
> single request/response. I have also provided a MAD output between
> Solaris IPoIB driver and IBSRM single GetTable request response below
> this example.
>
> Here is the MAD trace between solaris and OpenSM:
> Outgoing MAD:
> BaseVersion: 0x1
> MgmtClass: 0x3 - SubnAdm
> ClassVersion: 0x2
> R_Method: 0x12 - SubnAdmGetTable()
> Status: 0x0 - NO_ERROR
> ClassSpecific: 0x0
> TransactionID: 0x97651d1000000ec
> AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
>
> 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> 0: 01 03 02 12 00 00 00 00 09 76 51 d1 00 00 00 ec .........vQ.....
> 10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 .8..............
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 30: 00 00 00 00 00 00 80 b4 00 00 00 00 00 00 00 00 ................
> 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 50: 00 00 00 00 00 00 00 00 00 00 0b 1b 00 00 84 00 ................
> 60: ff ff 00 00 00 00 00 00 20 00 00 00 00 00 00 00 ........ .......
> 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> Incoming MAD:
> BaseVersion: 0x1
> MgmtClass: 0x3 - SubnAdm
> ClassVersion: 0x2
> R_Method: 0x92 -
> Status: 0x0 - NO_ERROR
> ClassSpecific: 0x0
> TransactionID: 0x97651d1000000ec
> AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
>
> 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> 0: 01 03 02 92 00 00 00 00 09 76 51 d1 00 00 00 ec .........vQ.....
> 10: 00 38 00 00 ff ff ff ff 01 01 77 00 00 00 00 01 .8........w.....
> 20: 00 00 00 14 00 00 00 00 00 00 00 00 00 07 00 00 ................
> 30: 00 00 00 00 00 00 80 b4 00 00 00 00 00 00 00 00 ................
> 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
It is likely failing the component checking in
osm_sa_mcmember_record.c::__osm_sa_mcm_by_comp_mask_cb due to an endian
issue. Either you can debug this code or I will early next week.
The component mask in the request is 0x80b4 so the only components
checked are QKey (0xb1b), MTU (exactly 2048 (4)), PKey (0xffff), and
scope (2).
If I don't hear anything by next week, I will work on this then.
Thanks.
-- Hal
> Here is the transaction between IBSRM and Solaris IPoIB driver.
>
> Outgoing MAD:
> BaseVersion: 0x1
> MgmtClass: 0x3 - SubnAdm
> ClassVersion: 0x2
> R_Method: 0x12 - SubnAdmGetTable()
> Status: 0x0 - NO_ERROR
> ClassSpecific: 0x0
> TransactionID: 0x8fecc610000009a
> AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
>
> 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> 0: 01 03 02 12 00 00 00 00 08 fe cc 61 00 00 00 9a ...........a....
> 10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 .8..............
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 30: 00 00 00 00 00 00 80 b4 00 00 00 00 00 00 00 00 ................
> 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 50: 00 00 00 00 00 00 00 00 81 23 45 68 00 00 84 00 .........#Eh....
> 60: 80 01 00 00 00 00 00 00 20 00 00 00 00 00 00 00 ........ .......
> 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> Incoming MAD:
> BaseVersion: 0x1
> MgmtClass: 0x3 - SubnAdm
> ClassVersion: 0x2
> R_Method: 0x92 -
> Status: 0x0 - NO_ERROR
> ClassSpecific: 0x0
> TransactionID: 0x8fecc610000009a
> AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
>
> 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
> 0: 01 03 02 92 00 00 00 00 08 fe cc 61 00 00 00 9a ...........a....
> 10: 00 38 00 00 00 00 00 00 01 01 73 00 00 00 00 01 .8........s.....
> 20: 00 00 01 40 00 00 00 00 00 00 00 00 00 07 00 00 ... at ............
> 30: 00 00 00 00 00 00 00 00 ff 12 40 1b 80 01 00 00 .......... at .....
> 40: 00 00 00 00 00 00 00 09 00 00 00 00 00 00 00 00 ................
> 50: 00 00 00 00 00 00 00 00 81 23 45 68 c0 04 84 00 .........#Eh....
> 60: 80 01 83 8d 00 00 00 00 20 00 00 00 00 00 00 00 ........ .......
> 70: ff 12 40 1b 80 01 00 00 00 00 00 00 00 00 00 01 .. at .............
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 90: 81 23 45 68 c0 03 84 00 80 01 83 8d 00 00 00 00 .#Eh............
> a0: 20 00 00 00 00 00 00 00 ff 12 40 1b 80 01 00 00 ......... at .....
> b0: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 ................
> c0: 00 00 00 00 00 00 00 00 81 23 45 68 c0 00 84 00 .........#Eh....
> d0: 80 01 83 8d 00 00 00 00 20 00 00 00 00 00 00 00 ........ .......
> e0: ff 12 60 1b 80 01 00 00 00 00 00 01 ff 76 5b 01 ..`..........v[.
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>
> Thanks
> Nitin
More information about the general
mailing list