[openib-general] Cray XD1 and OpenSM.. (ignoreing certain guids?)

[Hal Rosenstock]

On Fri, 2005-10-14 at 21:48, Hal Rosenstock wrote:
> On Fri, 2005-10-14 at 19:19, Troy Benjegerdes wrote:
> > In the interest of plugging absolutely everything I have with infiniband
> > ports together and seeing what falls over, I connected a Cray XD1 to a
> > small (2 machine) infiniband network running OpenSM.
> > 
> > Ideally, I'd like to find out what sort of minimal emulation code needs
> > to be running on the XD1 nodes to answer standards compliant SM
> > requests. 
> 
> You need an SMA on the Cray node. 
> 
> > Failing that, Is there a way we can tell OpenSM to ignore parts 
> > of the IB network? I am seeing a lot of stuff in the osm.log like this:
> > 
> > (port 16 is on the internal mellanox chip in the XD1)
> > 
> > ct 14 18:07:22 646959 [43005960] -> umad_receiver: ERR 5409: send
> > completed with error (method=1 attr=15) -- dropping.
> > Oct 14 18:07:22 646972 [43005960] -> umad_receiver: ERR 5411: DR SMP hop
> > ptr 0 hop count 3 DR SLID 0x0 DR DLID 0x0
> > Oct 14 18:07:22 646983 [43005960] -> __osm_sm_mad_ctrl_send_err_cb: ERR
> > 3113: MAD completed in error (IB_TIMEOUT).
> > Oct 14 18:07:22 647020 [43005960] -> SMP dump:
> >                                 base_ver................0x1
> >                                 mgmt_class..............0x81
> >                                 class_ver...............0x1
> >                                 method..................0x1 (SubnGet)
> >                                 D bit...................0x0
> >                                 status..................0x0
> >                                 hop_ptr.................0x0
> >                                 hop_count...............0x3
> >                                 trans_id................0x1420
> >                                 attr_id.................0x15 (PortInfo)
> >                                 resv....................0x0
> >                                 attr_mod................0x0
> >                                 m_key...................0x0000000000000000
> >                                 dr_slid.................0xFFFF
> >                                 dr_dlid.................0xFFFF
> > 
> >                                 Initial path: [0][1][4][16]
> >                                 Return path:  [0][0][0][0]
> >                                 Reserved:     [0][0][0][0][0][0][0]
> > 
> >                                 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00
> > 
> >                                 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00
> > 
> >                                 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00
> > 
> >                                 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00
> 
> I'm unaware of such an option. Not sure how you would specify which
> nodes to ignore. Why would you want them on the net if they are to be
> ignored ? 
> 
> Nodes are supposed to be IB compliant: SMA is a required component of
> all nodes.

So I presume there is no SMA for the Cray XD1. If someone is going to
implement this, we can document what portion of the SMA needs to
implemented to work with OpenSM. That wouldn't necessarily guarantee it
should work with any SM as other SMs may rely on some slightly different
things or do things in a slightly different way since there is much more
flexibility allowed on the SM side.

-- Hal