[Fwd: Re: [openib-general] Solaris IPoIB MTU with OpenSM]

Hal Rosenstock halr at voltaire.com
Fri Mar 4 12:53:22 PST 2005


Hi again Nitin,

Finally got a chance to work on this. I have a workaround for you for
now. Real patch later... Let me know if this does the trick for you. It
did for me.

-- Hal

Index: osm_sa_mcmember_record.c
===================================================================
--- osm_sa_mcmember_record.c	(revision 1953)
+++ osm_sa_mcmember_record.c	(working copy)
@@ -1522,9 +1522,11 @@
   if ((IB_MCR_COMPMASK_PROXY & comp_mask) &&
       (p_rcvd_rec->proxy_join != p_mgrp->mcmember_rec.proxy_join)) goto Exit;
 
+#if 0
   /* if defined MUST match exactly !*/
   if ((IB_MCR_COMPMASK_MTU_SEL & comp_mask) &&
       ((p_rcvd_rec->mtu >> 6) != (p_mgrp->mcmember_rec.mtu >> 6))) goto Exit;
+#endif
 
   if ((IB_MCR_COMPMASK_MTU & comp_mask) &&
       ((p_rcvd_rec->mtu & 0x3F) != (p_mgrp->mcmember_rec.mtu & 0x3F))) goto Exit;



-----Forwarded Message-----

From: Hal Rosenstock <halr at voltaire.com>
To: Nitin Hande <Nitin.Hande at Sun.COM>
Cc: openib <openib-general at openib.org>, Tom Duffy <Tom.Duffy at Sun.COM>
Subject: Re: [openib-general] Solaris IPoIB MTU with OpenSM
Date: 24 Feb 2005 08:42:23 -0500

Hi Nitin,

On Wed, 2005-02-23 at 17:19, Nitin Hande wrote:
> Hal, 
> 
> [comments below]
> On Wed, 2005-02-23 at 02:19, Hal Rosenstock wrote:
> > On Tue, 2005-02-22 at 22:56, Nitin Hande wrote:
> > > So I tried the latest patches and preliminarily things seem to be
> > > working fine. 
> > 
> > Yipee.
> [snip..]
> > 
> > > 
> > > So after this test above, I try to run snoop on the solaris interface
> > > and get the following error message from the layer below IPoIB:
> > > 
> > > Feb 22 19:50:25 dongon.SFBay.Sun.COM ibd: [ID 517869 kern.info] NOTICE:
> > > ibd0: HCA GUID 0002c901097651d0 port 1 PKEY ffff Could not get list of
> > > IBA multicast groups
> > > 
> > > My preliminary assumption is that OpenSm is not returning the list of
> > > multicast groups that the ibd interface has joined. I will look at the
> > > MAD's tomorrow and try to ascertain that.
> > 
> > How does S10 request this ? Remember that if it is a GetTable and
> > doesn't fit in a single MAD, it will be broken now. If that is the case,
> > we will live with this until we have real RMPP.
> Below is an an example of a single GetTable request and response between
> Solaris and OpenSM. OpenSM is not reporting the MCgroups in case of a
> single request/response.  I have also provided a MAD output between
> Solaris IPoIB driver and IBSRM single GetTable request response below
> this example.
> 
> Here is the MAD trace between solaris and OpenSM:
> Outgoing MAD:
>         BaseVersion: 0x1
>         MgmtClass: 0x3 - SubnAdm
>         ClassVersion: 0x2
>         R_Method: 0x12 - SubnAdmGetTable()
>         Status: 0x0 - NO_ERROR
>         ClassSpecific: 0x0
>         TransactionID: 0x97651d1000000ec
>         AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> 
>      0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
>  0: 01 03 02 12 00 00 00 00 09 76 51 d1 00 00 00 ec  .........vQ.....
> 10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  .8..............
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 30: 00 00 00 00 00 00 80 b4 00 00 00 00 00 00 00 00  ................
> 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 50: 00 00 00 00 00 00 00 00 00 00 0b 1b 00 00 84 00  ................
> 60: ff ff 00 00 00 00 00 00 20 00 00 00 00 00 00 00  ........ .......
> 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> Incoming MAD:
>         BaseVersion: 0x1
>         MgmtClass: 0x3 - SubnAdm
>         ClassVersion: 0x2
>         R_Method: 0x92 -
>         Status: 0x0 - NO_ERROR
>         ClassSpecific: 0x0
>         TransactionID: 0x97651d1000000ec
>         AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> 
>      0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
>  0: 01 03 02 92 00 00 00 00 09 76 51 d1 00 00 00 ec  .........vQ.....
> 10: 00 38 00 00 ff ff ff ff 01 01 77 00 00 00 00 01  .8........w.....
> 20: 00 00 00 14 00 00 00 00 00 00 00 00 00 07 00 00  ................
> 30: 00 00 00 00 00 00 80 b4 00 00 00 00 00 00 00 00  ................
> 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................

It is likely failing the component checking in
osm_sa_mcmember_record.c::__osm_sa_mcm_by_comp_mask_cb due to an endian
issue. Either you can debug this code or I will early next week.

The component mask in the request is 0x80b4 so the only components
checked are QKey (0xb1b), MTU (exactly 2048 (4)), PKey (0xffff), and
scope (2).

If I don't hear anything by next week, I will work on this then.

Thanks.

-- Hal

> Here is the transaction between IBSRM and Solaris IPoIB driver. 
> 
> Outgoing MAD:
>         BaseVersion: 0x1
>         MgmtClass: 0x3 - SubnAdm
>         ClassVersion: 0x2
>         R_Method: 0x12 - SubnAdmGetTable()
>         Status: 0x0 - NO_ERROR
>         ClassSpecific: 0x0
>         TransactionID: 0x8fecc610000009a
>         AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> 
>      0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
>  0: 01 03 02 12 00 00 00 00 08 fe cc 61 00 00 00 9a  ...........a....
> 10: 00 38 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  .8..............
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 30: 00 00 00 00 00 00 80 b4 00 00 00 00 00 00 00 00  ................
> 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 50: 00 00 00 00 00 00 00 00 81 23 45 68 00 00 84 00  .........#Eh....
> 60: 80 01 00 00 00 00 00 00 20 00 00 00 00 00 00 00  ........ .......
> 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> Incoming MAD:
>         BaseVersion: 0x1
>         MgmtClass: 0x3 - SubnAdm
>         ClassVersion: 0x2
>         R_Method: 0x92 -
>         Status: 0x0 - NO_ERROR
>         ClassSpecific: 0x0
>         TransactionID: 0x8fecc610000009a
>         AttributeID: 0x38 - SA_MCMEMBERRECORD_ATTRID
> 
>      0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  0123456789abcdef
>  0: 01 03 02 92 00 00 00 00 08 fe cc 61 00 00 00 9a  ...........a....
> 10: 00 38 00 00 00 00 00 00 01 01 73 00 00 00 00 01  .8........s.....
> 20: 00 00 01 40 00 00 00 00 00 00 00 00 00 07 00 00  ... at ............
> 30: 00 00 00 00 00 00 00 00 ff 12 40 1b 80 01 00 00  .......... at .....
> 40: 00 00 00 00 00 00 00 09 00 00 00 00 00 00 00 00  ................
> 50: 00 00 00 00 00 00 00 00 81 23 45 68 c0 04 84 00  .........#Eh....
> 60: 80 01 83 8d 00 00 00 00 20 00 00 00 00 00 00 00  ........ .......
> 70: ff 12 40 1b 80 01 00 00 00 00 00 00 00 00 00 01  .. at .............
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 90: 81 23 45 68 c0 03 84 00 80 01 83 8d 00 00 00 00  .#Eh............
> a0: 20 00 00 00 00 00 00 00 ff 12 40 1b 80 01 00 00   ......... at .....
> b0: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00  ................
> c0: 00 00 00 00 00 00 00 00 81 23 45 68 c0 00 84 00  .........#Eh....
> d0: 80 01 83 8d 00 00 00 00 20 00 00 00 00 00 00 00  ........ .......
> e0: ff 12 60 1b 80 01 00 00 00 00 00 01 ff 76 5b 01  ..`..........v[.
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> 
> Thanks
> Nitin





More information about the general mailing list