[openib-general] [PATCH] osm: handle local events

[Sasha Khapyorsky]

On 21:41 Sat 26 Aug     , Michael S. Tsirkin wrote:
> Quoting r. Sasha Khapyorsky <sashak at voltaire.com>:
> > Subject: Re: [openib-general] [PATCH] osm: handle local events
> > 
> > On 16:28 Thu 24 Aug     , Michael S. Tsirkin wrote:
> > > Quoting r. Yevgeny Kliteynik <kliteyn at mellanox.co.il>:
> > > > Index: libvendor/osm_vendor_ibumad.c
> > > > ===================================================================
> > > > --- libvendor/osm_vendor_ibumad.c	(revision 8998)
> > > > +++ libvendor/osm_vendor_ibumad.c	(working copy)
> > > > @@ -72,6 +72,7 @@
> > > >  #include <opensm/osm_log.h>
> > > >  #include <opensm/osm_mad_pool.h>
> > > >  #include <vendor/osm_vendor_api.h>
> > > > +#include <infiniband/verbs.h>
> > > >  
> > > >  /****s* OpenSM: Vendor AL/osm_umad_bind_info_t
> > > >   * NAME
> > > 
> > > NAK.
> > > 
> > > This means that the SM becomes dependent on the uverbs module.  I don't think
> > > this is a good idea.  Let's not go there - SM should depend just on the umad
> > > module and libc.
> > 
> > Agree on this point. I dislike this new libibverbs dependency too. I
> > think we need to work with umad.
> > 
> > So more generic question: some application performs blocked read() from
> > /dev/umadN, should this read() be interrupted and return error (with
> > appropriate errno value), then the port state becomes DOWN?
> > I think yes, it should. Other opinions? Sean?
> 
> One thing seems obvious: if device goes away it seems obvious that we should
> return ENODEV from any read. Isn't this already done?

I think it does nothing when the port becomes down (or the driver sends
PORT_ERR event).

> 
> > 
> > And if yes, then in OpenSM we will need just to check errno value upon
> > umad_recv() failure.
> > 
> > Sasha
> 
> Might be a good idea. Hoever, such an approach by default is an ABI change so it
> could break some apps. 

It should not change ABI, only slight change for read()/write()
behaviors. I think more important question is "is proposed umad behavior
right?"

> Could this be made an option somehow?

With fcntl() (or similar)?

> Think also about
> a race where I read *after* the state was changed to DOWN.

Which race?

If blocked read() starts on DOWN port we may decide how to handle this,
I may think about two options: or return error, or work as usual.. Both
may be acceptable (but later does not required any changes).

> 
> Another question comes to mind:
> does not SM care about physical link state changes as well?

Not explicitly AFAIK.

> Assuming I disconnect and re-connect the cable, does not
> SM want to know and try bringing the logical link up?

AFAIK OpenSM does not detect the local port disconnection explicitly
(however will detect this at next periodic sweep if such configured), but
it will get report from the switch when the local port will be
re-connected back.

Sasha

> How is this handled currently? Can the same mechanism be used
> for port state events?
> 
> -- 
> MST