[openib-general] RE: [PATCH] RFC: opensm: serialize osm_state_mgr_process()
Eitan Zahavi
eitan at mellanox.co.il
Mon May 22 04:44:21 PDT 2006
Hi Sasha,
> >
> > The idea to use pthread in OpenSM code is totally wrong.
> > Please stop doing this. We want this code to be shared with Windows
and
> > this breaks it.
>
> The problem is that complib does not provide needed primitives, mostly
> in synchronization and thread management areas (like
pthread_cond_wait()
> and friends, pthread_cancel() and friends).
[EZ] The problem is that any code you write in OpenSM should be portable
to existing complib such that it would be portable to windows.
If some thread interface is missing (I am surprised we could get so far
without that primitive) - please add to complib and also make sure
Windows native threads can support that primitive.
>
> The option to extend complib also does not look very helpful for
Windows
> too - complib uses pthreads as backend anyway.
[EZ] No the windows complib does not use pthread at all.
>
> Is using of pthread library with Windows may solve sharing issue?
[EZ] It might if you are willing to go through this un-needed exercise
...
We have a complib in windows that is well tested and functional today.
I do not see why you need to re-write it.
>
> Other option I may think about is to use pthread wrapper (in the same
> way as it is done today with complib).
[EZ] This does not solve the need to implement the same functionality in
windows.
And since I do not want to re-implement complib for windows - I prefer
sticking with complib wrappers.
>
> >
> > Also please provide a clear RFC for what this patch is trying to do.
>
> Basically this serializes execution of osm_state_mgr_process(), so
> instead of to be directly called via dispatcher's callback (with
possible
> waiting for the lock), state_manager will be signaled (and wakeuped if
> necessary), as result we don't need big state_lock anymore, mutex is
> needed only to protect signal_mask (opensm's osm_signal_t, not *nix
> signals) update.
[EZ] I do not see what problem you are trying to solve?
What is wrong with the current implementation? Are you fixing a bug?
The dispatching mechanism used allows passing the state manager signals
which must be processed in order. The state_lock guarantees this order.
>
> Sasha
>
> > Eitan Zahavi
> > Senior Engineering Director, Software Architect
> > Mellanox Technologies LTD
> > Tel:+972-4-9097208
> > Fax:+972-4-9593245
> > P.O. Box 586 Yokneam 20692 ISRAEL
> >
> >
> > > -----Original Message-----
> > > From: Sasha Khapyorsky [mailto:sashak at voltaire.com]
> > > Sent: Monday, May 22, 2006 2:02 AM
> > > To: Hal Rosenstock; openib-general at openib.org
> > > Cc: Eitan Zahavi; Yael Kalka; Ofer Gigi; sashak at voltaire.com;
> > elid at voltaire.com
> > > Subject: [PATCH] RFC: opensm: serialize osm_state_mgr_process()
> > >
> > > Hello,
> > >
> > > Please comment (and test).
> > >
> > > Thanks,
> > > Sasha.
> > >
> > > This serializes execution of osm_state_mgr_process() and removes
the
> > big
> > > state_lock. This should reduce "locked state" time and prevent
> > potential
> > > dispatcher blocking.
> > >
> > > Other important change here is direct usage of pthread primitives
> > > instead of "traditional" cl_thread* stuff.
> > >
> > > Signed-off-by: Sasha Khapyorsky <sashak at voltaire.com>
> > > ---
> > >
> > > osm/include/Makefile.am | 1
> > > osm/include/opensm/osm_msgdef.h | 15 --
> > > osm/include/opensm/osm_sm.h | 36 ++++-
> > > osm/include/opensm/osm_state_mgr.h | 4 -
> > > osm/include/opensm/osm_state_mgr_ctrl.h | 236
> > -------------------------------
> > > osm/opensm/Makefile.am | 5 -
> > > osm/opensm/osm_helper.c | 1
> > > osm/opensm/osm_node_info_rcv.c | 5 -
> > > osm/opensm/osm_port_info_rcv.c | 3
> > > osm/opensm/osm_sm.c | 160
+++++++++++----------
> > > osm/opensm/osm_sm_mad_ctrl.c | 26 ---
> > > osm/opensm/osm_sm_state_mgr.c | 8 +
> > > osm/opensm/osm_sminfo_rcv.c | 8 +
> > > osm/opensm/osm_state_mgr.c | 24 ---
> > > osm/opensm/osm_state_mgr_ctrl.c | 132 -----------------
> > > osm/opensm/osm_sw_info_rcv.c | 3
> > > osm/opensm/osm_sweep_fail_ctrl.c | 4 -
> > > osm/opensm/osm_trap_rcv.c | 3
> > > osm/opensm/osm_vl15intf.c | 25 ---
> > > 19 files changed, 143 insertions(+), 556 deletions(-)
> > >
> > > diff --git a/osm/include/Makefile.am b/osm/include/Makefile.am
> > > index 2bee762..0af78a0 100644
> > > --- a/osm/include/Makefile.am
> > > +++ b/osm/include/Makefile.am
> > > @@ -120,7 +120,6 @@ EXTRA_DIST = \
> > > $(srcdir)/opensm/osm_vl15intf.h \
> > > $(srcdir)/opensm/osm_drop_mgr.h \
> > > $(srcdir)/opensm/osm_port_info_rcv.h \
> > > - $(srcdir)/opensm/osm_state_mgr_ctrl.h \
> > > $(srcdir)/complib/cl_thread_osd.h \
> > > $(srcdir)/complib/cl_packon.h \
> > > $(srcdir)/complib/cl_atomic_osd.h \
> > > diff --git a/osm/include/opensm/osm_msgdef.h
> > b/osm/include/opensm/osm_msgdef.h
> > > index a1b5743..6956c86 100644
> > > --- a/osm/include/opensm/osm_msgdef.h
> > > +++ b/osm/include/opensm/osm_msgdef.h
> > > @@ -148,20 +148,6 @@ BEGIN_C_DECLS
> > > *
> > > * SOURCE
> > > ***********/
> > > -/****d* OpenSM: Dispatcher Messages/OSM_MSG_NO_SMPS_OUTSTANDING
> > > -* NAME
> > > -* OSM_MSG_NO_SMPS_OUTSTANDING
> > > -*
> > > -* DESCRIPTION
> > > -* Message indicating that there are no outstanding SMPs on the
> > subnet.
> > > -*
> > > -* NOTES
> > > -* Sent by: osm_mad_ctrl_t
> > > -* Received by: osm_state_mgr_ctrl_t
> > > -* Delivery notice: no
> > > -*
> > > -* SOURCE
> > > -***********/
> > > enum
> > > {
> > > OSM_MSG_REQ = 0,
> > > @@ -169,7 +155,6 @@ enum
> > > OSM_MSG_MAD_PORT_INFO,
> > > OSM_MSG_MAD_SWITCH_INFO,
> > > OSM_MSG_MAD_NODE_DESC,
> > > - OSM_MSG_NO_SMPS_OUTSTANDING,
> > > OSM_MSG_MAD_NODE_RECORD,
> > > OSM_MSG_MAD_PORTINFO_RECORD,
> > > OSM_MSG_MAD_SERVICE_RECORD,
> > > diff --git a/osm/include/opensm/osm_sm.h
b/osm/include/opensm/osm_sm.h
> > > index d6086d4..def43c8 100644
> > > --- a/osm/include/opensm/osm_sm.h
> > > +++ b/osm/include/opensm/osm_sm.h
> > > @@ -69,7 +69,6 @@ #include <opensm/osm_port_info_rcv_ctrl.
> > > #include <opensm/osm_sw_info_rcv_ctrl.h>
> > > #include <opensm/osm_node_desc_rcv_ctrl.h>
> > > #include <opensm/osm_sm_mad_ctrl.h>
> > > -#include <opensm/osm_state_mgr_ctrl.h>
> > > #include <opensm/osm_lid_mgr.h>
> > > #include <opensm/osm_ucast_mgr.h>
> > > #include <opensm/osm_link_mgr.h>
> > > @@ -131,7 +130,6 @@ BEGIN_C_DECLS
> > > typedef struct _osm_sm
> > > {
> > > osm_thread_state_t thread_state;
> > > - cl_event_t signal;
> > > cl_event_t subnet_up_event;
> > > cl_thread_t sweeper;
> > > osm_subn_t *p_subn;
> > > @@ -143,6 +141,9 @@ typedef struct _osm_sm
> > > cl_dispatcher_t *p_disp;
> > > cl_plock_t *p_lock;
> > > atomic32_t sm_trans_id;
> > > + unsigned signal_mask;
> > > + pthread_mutex_t mutex;
> > > + pthread_cond_t cond;
> > > osm_req_t req;
> > > osm_req_ctrl_t req_ctrl;
> > > osm_resp_t resp;
> > > @@ -155,7 +156,6 @@ typedef struct _osm_sm
> > > osm_sm_mad_ctrl_t mad_ctrl;
> > > osm_si_rcv_t si_rcv;
> > > osm_si_rcv_ctrl_t si_rcv_ctrl;
> > > - osm_state_mgr_ctrl_t state_mgr_ctrl;
> > > osm_lid_mgr_t lid_mgr;
> > > osm_ucast_mgr_t ucast_mgr;
> > > osm_link_mgr_t link_mgr;
> > > @@ -387,6 +387,33 @@ osm_sm_init(
> > > * SM object, osm_sm_construct, osm_sm_destroy
> > > *********/
> > >
> > > +/****f* OpenSM: SM/osm_sm_signal
> > > +* NAME
> > > +* osm_sm_signal
> > > +*
> > > +* DESCRIPTION
> > > +* Forward signal to state engine
> > > +*
> > > +* SYNOPSIS
> > > +*/
> > > +void
> > > +osm_sm_signal(
> > > + IN osm_sm_t* const p_sm,
> > > + IN osm_signal_t signal );
> > > +/*
> > > +* PARAMETERS
> > > +* p_sm
> > > +* [in] Pointer to an osm_sm_t object.
> > > +*
> > > +* signal
> > > +* [in] Signal to the state engine.
> > > +*
> > > +* NOTES
> > > +*
> > > +* SEE ALSO
> > > +* SM object
> > > +*********/
> > > +
> > > /****f* OpenSM: SM/osm_sm_sweep
> > > * NAME
> > > * osm_sm_sweep
> > > @@ -404,9 +431,6 @@ osm_sm_sweep(
> > > * p_sm
> > > * [in] Pointer to an osm_sm_t object.
> > > *
> > > -* RETURN VALUES
> > > -* IB_SUCCESS if the sweep completed successfully.
> > > -*
> > > * NOTES
> > > *
> > > * SEE ALSO
> > > diff --git a/osm/include/opensm/osm_state_mgr.h
> > > b/osm/include/opensm/osm_state_mgr.h
> > > index ad4afa0..5e76463 100644
> > > --- a/osm/include/opensm/osm_state_mgr.h
> > > +++ b/osm/include/opensm/osm_state_mgr.h
> > > @@ -116,7 +116,6 @@ typedef struct _osm_state_mgr
> > > osm_stats_t *p_stats;
> > > struct _osm_sm_state_mgr *p_sm_state_mgr;
> > > const osm_sm_mad_ctrl_t *p_mad_ctrl;
> > > - cl_spinlock_t state_lock;
> > > cl_spinlock_t idle_lock;
> > > cl_qlist_t idle_time_list;
> > > cl_plock_t *p_lock;
> > > @@ -161,9 +160,6 @@ typedef struct _osm_state_mgr
> > > * p_mad_ctrl
> > > * Pointer to the SM's MAD Controller object.
> > > *
> > > -* state_lock
> > > -* Spinlock guarding the state and processes.
> > > -*
> > > * p_lock
> > > * lock guarding the subnet object.
> > > *
> > > diff --git a/osm/include/opensm/osm_state_mgr_ctrl.h
> > > b/osm/include/opensm/osm_state_mgr_ctrl.h
> > > deleted file mode 100644
> > > index 9ffcfb0..0000000
> > > --- a/osm/include/opensm/osm_state_mgr_ctrl.h
> > > +++ /dev/null
> > > @@ -1,236 +0,0 @@
> > > -/*
> > > - * Copyright (c) 2004, 2005 Voltaire, Inc. All rights reserved.
> > > - * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights
> > reserved.
> > > - * Copyright (c) 1996-2003 Intel Corporation. All rights
reserved.
> > > - *
> > > - * This software is available to you under a choice of one of two
> > > - * licenses. You may choose to be licensed under the terms of
the
> > GNU
> > > - * General Public License (GPL) Version 2, available from the
file
> > > - * COPYING in the main directory of this source tree, or the
> > > - * OpenIB.org BSD license below:
> > > - *
> > > - * Redistribution and use in source and binary forms, with or
> > > - * without modification, are permitted provided that the
> > following
> > > - * conditions are met:
> > > - *
> > > - * - Redistributions of source code must retain the above
> > > - * copyright notice, this list of conditions and the
following
> > > - * disclaimer.
> > > - *
> > > - * - Redistributions in binary form must reproduce the above
> > > - * copyright notice, this list of conditions and the
following
> > > - * disclaimer in the documentation and/or other materials
> > > - * provided with the distribution.
> > > - *
> > > - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
> KIND,
> > > - * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
> > > WARRANTIES OF
> > > - * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
> > > - * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
> COPYRIGHT
> > > HOLDERS
> > > - * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
> IN
> > > AN
> > > - * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF
> OR
> > > IN
> > > - * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> IN
> > > THE
> > > - * SOFTWARE.
> > > - *
> > > - * $Id$
> > > - */
> > > -
> > > -
> > > -/*
> > > - * Abstract:
> > > - * Declaration of osm_state_mgr_ctrl_t.
> > > - * This object represents a controller that receives the
> > > - * State indication after a subnet sweep.
> > > - * This object is part of the OpenSM family of objects.
> > > - *
> > > - * Environment:
> > > - * Linux User Mode
> > > - *
> > > - * $Revision: 1.4 $
> > > - */
> > > -
> > > -#ifndef _OSM_STATE_MGR_CTRL_H_
> > > -#define _OSM_STATE_MGR_CTRL_H_
> > > -
> > > -
> > > -#include <complib/cl_dispatcher.h>
> > > -#include <opensm/osm_base.h>
> > > -#include <opensm/osm_state_mgr.h>
> > > -#include <opensm/osm_log.h>
> > > -
> > > -#ifdef __cplusplus
> > > -# define BEGIN_C_DECLS extern "C" {
> > > -# define END_C_DECLS }
> > > -#else /* !__cplusplus */
> > > -# define BEGIN_C_DECLS
> > > -# define END_C_DECLS
> > > -#endif /* __cplusplus */
> > > -
> > > -BEGIN_C_DECLS
> > > -
> > > -/****h* OpenSM/State Manager Controller
> > > -* NAME
> > > -* State Manager Controller
> > > -*
> > > -* DESCRIPTION
> > > -* The State Manager Controller object encapsulates the
information
> > > -* needed to pass the dispatcher message from the
dispatcher
> > > -* to the State Manager.
> > > -*
> > > -* The State Manager Controller object is thread safe.
> > > -*
> > > -* This object should be treated as opaque and should be
> > > -* manipulated only through the provided functions.
> > > -*
> > > -* AUTHOR
> > > -* Steve King, Intel
> > > -*
> > > -*********/
> > > -/****s* OpenSM: State Manager Controller/osm_state_mgr_ctrl_t
> > > -* NAME
> > > -* osm_state_mgr_ctrl_t
> > > -*
> > > -* DESCRIPTION
> > > -* State Manager Controller structure.
> > > -*
> > > -* This object should be treated as opaque and should
> > > -* be manipulated only through the provided functions.
> > > -*
> > > -* SYNOPSIS
> > > -*/
> > > -typedef struct _osm_state_mgr_ctrl
> > > -{
> > > - osm_state_mgr_t *p_mgr;
> > > - osm_log_t *p_log;
> > > - cl_dispatcher_t *p_disp;
> > > - cl_disp_reg_handle_t h_disp;
> > > -
> > > -} osm_state_mgr_ctrl_t;
> > > -/*
> > > -* FIELDS
> > > -* p_mgr
> > > -* Pointer to the State Manager object.
> > > -*
> > > -* p_log
> > > -* Pointer to the log object.
> > > -*
> > > -* p_disp
> > > -* Pointer to the Dispatcher.
> > > -*
> > > -* h_disp
> > > -* Handle returned from dispatcher registration.
> > > -*
> > > -* SEE ALSO
> > > -* State Manager Controller object
> > > -*********/
> > > -
> > > -/****f* OpenSM: State Manager
Controller/osm_state_mgr_ctrl_construct
> > > -* NAME
> > > -* osm_state_mgr_ctrl_construct
> > > -*
> > > -* DESCRIPTION
> > > -* This function constructs a State Manager Controller
object.
> > > -*
> > > -* SYNOPSIS
> > > -*/
> > > -void
> > > -osm_state_mgr_ctrl_construct(
> > > - IN osm_state_mgr_ctrl_t* const p_ctrl );
> > > -/*
> > > -* PARAMETERS
> > > -* p_ctrl
> > > -* [in] Pointer to a State Manager Controller
> > > -* object to construct.
> > > -*
> > > -* RETURN VALUE
> > > -* This function does not return a value.
> > > -*
> > > -* NOTES
> > > -* Allows calling osm_state_mgr_ctrl_init,
> > osm_state_mgr_ctrl_destroy,
> > > -* and osm_state_mgr_ctrl_is_inited.
> > > -*
> > > -* Calling osm_state_mgr_ctrl_construct is a prerequisite
to
> > calling any other
> > > -* method except osm_state_mgr_ctrl_init.
> > > -*
> > > -* SEE ALSO
> > > -* State Manager Controller object,
osm_state_mgr_ctrl_init,
> > > -* osm_state_mgr_ctrl_destroy
> > > -*********/
> > > -
> > > -/****f* OpenSM: State Manager
Controller/osm_state_mgr_ctrl_destroy
> > > -* NAME
> > > -* osm_state_mgr_ctrl_destroy
> > > -*
> > > -* DESCRIPTION
> > > -* The osm_state_mgr_ctrl_destroy function destroys the
object,
> > releasing
> > > -* all resources.
> > > -*
> > > -* SYNOPSIS
> > > -*/
> > > -void
> > > -osm_state_mgr_ctrl_destroy(
> > > - IN osm_state_mgr_ctrl_t* const p_ctrl );
> > > -/*
> > > -* PARAMETERS
> > > -* p_ctrl
> > > -* [in] Pointer to the object to destroy.
> > > -*
> > > -* RETURN VALUE
> > > -* This function does not return a value.
> > > -*
> > > -* NOTES
> > > -* Performs any necessary cleanup of the specified
> > > -* State Manager Controller object.
> > > -* Further operations should not be attempted on the
destroyed
> > object.
> > > -* This function should only be called after a call to
> > > -* osm_state_mgr_ctrl_construct or osm_state_mgr_ctrl_init.
> > > -*
> > > -* SEE ALSO
> > > -* State Manager Controller object,
osm_state_mgr_ctrl_construct,
> > > -* osm_state_mgr_ctrl_init
> > > -*********/
> > > -
> > > -/****f* OpenSM: State Manager Controller/osm_state_mgr_ctrl_init
> > > -* NAME
> > > -* osm_state_mgr_ctrl_init
> > > -*
> > > -* DESCRIPTION
> > > -* The osm_state_mgr_ctrl_init function initializes a
> > > -* State Manager Controller object for use.
> > > -*
> > > -* SYNOPSIS
> > > -*/
> > > -ib_api_status_t
> > > -osm_state_mgr_ctrl_init(
> > > - IN osm_state_mgr_ctrl_t* const p_ctrl,
> > > - IN osm_state_mgr_t* const p_mgr,
> > > - IN osm_log_t* const p_log,
> > > - IN cl_dispatcher_t* const p_disp );
> > > -/*
> > > -* PARAMETERS
> > > -* p_ctrl
> > > -* [in] Pointer to an osm_state_mgr_ctrl_t object
to
> > initialize.
> > > -*
> > > -* p_mgr
> > > -* [in] Pointer to an osm_state_mgr_t object.
> > > -*
> > > -* p_log
> > > -* [in] Pointer to the log object.
> > > -*
> > > -* p_disp
> > > -* [in] Pointer to the OpenSM central Dispatcher.
> > > -*
> > > -* RETURN VALUES
> > > -* IB_SUCCESS if the State Manager Controller object
> > > -* was initialized successfully.
> > > -*
> > > -* NOTES
> > > -* Allows calling other State Manager Controller methods.
> > > -*
> > > -* SEE ALSO
> > > -* State Manager Controller object,
osm_state_mgr_ctrl_construct,
> > > -* osm_state_mgr_ctrl_destroy
> > > -*********/
> > > -
> > > -END_C_DECLS
> > > -
> > > -#endif /* OSM_STATE_MGR_CTRL_H_ */
> > > diff --git a/osm/opensm/Makefile.am b/osm/opensm/Makefile.am
> > > index 43fe8c1..7b1060a 100644
> > > --- a/osm/opensm/Makefile.am
> > > +++ b/osm/opensm/Makefile.am
> > > @@ -78,8 +78,7 @@ opensm_SOURCES = main.c osm_console.c os
> > > osm_slvl_map_rcv.c osm_slvl_map_rcv_ctrl.c \
> > > osm_sm.c osm_sminfo_rcv.c \
> > > osm_sminfo_rcv_ctrl.c osm_sm_mad_ctrl.c \
> > > - osm_sm_state_mgr.c osm_state_mgr.c \
> > > - osm_state_mgr_ctrl.c osm_subnet.c \
> > > + osm_sm_state_mgr.c osm_state_mgr.c osm_subnet.c \
> > > osm_sweep_fail_ctrl.c osm_sw_info_rcv.c \
> > > osm_sw_info_rcv_ctrl.c osm_switch.c \
> > > osm_prtn.c osm_prtn_config.c osm_qos.c \
> > > @@ -104,7 +103,7 @@ # we need to be able to load libraries f
> > > # we always give precedence to local tree libs and then use the
> > pre-installed ones.
> > > opensm_LDADD = -L../complib -L../libvendor -L. $(OSMV_LDADD)
-lopensm
> > -
> > > losmcomp -losmvendor
> > >
> > > -opensm_LDFLAGS = -Wl,--rpath -Wl,$(libdir) -lpthread
> > > +opensm_LDFLAGS = -Wl,--rpath -Wl,$(libdir) -lpthread -lrt
> > >
> > > opensmincludedir = $(includedir)/infiniband/opensm
> > >
> > > diff --git a/osm/opensm/osm_helper.c b/osm/opensm/osm_helper.c
> > > index 3886609..a966bbe 100644
> > > --- a/osm/opensm/osm_helper.c
> > > +++ b/osm/opensm/osm_helper.c
> > > @@ -1895,7 +1895,6 @@ static const char* const __osm_disp_msg_
> > > "OSM_MSG_MAD_PORT_INFO,",
> > > "OSM_MSG_MAD_SWITCH_INFO",
> > > "OSM_MSG_MAD_NODE_DESC",
> > > - "OSM_MSG_NO_SMPS_OUTSTANDING",
> > > "OSM_MSG_MAD_NODE_RECORD",
> > > "OSM_MSG_MAD_PORTINFO_RECORD",
> > > "OSM_MSG_MAD_SERVICE_RECORD",
> > > diff --git a/osm/opensm/osm_node_info_rcv.c
> > b/osm/opensm/osm_node_info_rcv.c
> > > index 59257a0..7ea4366 100644
> > > --- a/osm/opensm/osm_node_info_rcv.c
> > > +++ b/osm/opensm/osm_node_info_rcv.c
> > > @@ -69,6 +69,7 @@ #include <opensm/osm_subnet.h>
> > > #include <opensm/osm_mad_pool.h>
> > > #include <opensm/osm_helper.h>
> > > #include <opensm/osm_msgdef.h>
> > > +#include <opensm/osm_opensm.h>
> > >
> > >
> > >
> >
/**********************************************************************
> > > @@ -1088,11 +1089,11 @@ osm_ni_rcv_process(
> > >
> > > /*
> > > * If we processed a new node - need to signal to the state_mgr
> > that
> > > - * change detected. BUT - we cannot call the
osm_state_mgr_process
> > > + * change detected. BUT - we cannot call the osm_sm_signal
> > > * from within the lock of p_rcv->p_lock (can cause a
deadlock).
> > > */
> > > if ( process_new_flag )
> > > - osm_state_mgr_process( p_rcv->p_state_mgr,
> > > OSM_SIGNAL_CHANGE_DETECTED );
> > > + osm_sm_signal( &p_rcv->p_subn->p_osm->sm,
> > > OSM_SIGNAL_CHANGE_DETECTED );
> > >
> > > Exit:
> > > OSM_LOG_EXIT( p_rcv->p_log );
> > > diff --git a/osm/opensm/osm_port_info_rcv.c
> > b/osm/opensm/osm_port_info_rcv.c
> > > index a08c57c..7405ef0 100644
> > > --- a/osm/opensm/osm_port_info_rcv.c
> > > +++ b/osm/opensm/osm_port_info_rcv.c
> > > @@ -69,6 +69,7 @@ #include <opensm/osm_helper.h>
> > > #include <vendor/osm_vendor_api.h>
> > > #include <opensm/osm_pkey.h>
> > > #include <opensm/osm_remote_sm.h>
> > > +#include <opensm/osm_opensm.h>
> > >
> > >
> >
/**********************************************************************
> > >
> >
**********************************************************************/
> > > @@ -701,7 +702,7 @@ osm_pi_rcv_process(
> > > " port = %u, Commencing heavy sweep\n",
> > > cl_ntoh64( node_guid ),
> > > cl_ntoh64( port_guid ) );
> > > - osm_state_mgr_process( p_rcv->p_state_mgr,
> > > + osm_sm_signal( &p_rcv->p_subn->p_osm->sm,
> > > OSM_SIGNAL_CHANGE_DETECTED );
> > > goto Exit;
> > > }
> > > diff --git a/osm/opensm/osm_sm.c b/osm/opensm/osm_sm.c
> > > index 0e09f26..5b5eb3f 100644
> > > --- a/osm/opensm/osm_sm.c
> > > +++ b/osm/opensm/osm_sm.c
> > > @@ -55,6 +55,8 @@ #if HAVE_CONFIG_H
> > > # include <config.h>
> > > #endif /* HAVE_CONFIG_H */
> > >
> > > +#include <time.h>
> > > +#include <errno.h>
> > > #include <string.h>
> > > #include <complib/cl_memory.h>
> > > #include <complib/cl_qmap.h>
> > > @@ -79,53 +81,65 @@ void
> > > __osm_sm_sweeper(
> > > IN void *p_ptr )
> > > {
> > > - ib_api_status_t status;
> > > osm_sm_t *const p_sm = ( osm_sm_t * ) p_ptr;
> > > + unsigned i, signals;
> > >
> > > OSM_LOG_ENTER( p_sm->p_log, __osm_sm_sweeper );
> > >
> > > - if( p_sm->thread_state == OSM_THREAD_STATE_INIT )
> > > - {
> > > - p_sm->thread_state = OSM_THREAD_STATE_RUN;
> > > - }
> > > -
> > > - /* If the sweep interval was updated before - then run only if
> > > - * it is not zero. */
> > > - while( p_sm->thread_state == OSM_THREAD_STATE_RUN &&
> > > - p_sm->p_subn->opt.sweep_interval != 0 )
> > > - {
> > > - /* do the sweep only if we are in MASTER state */
> > > - if( p_sm->p_subn->sm_state == IB_SMINFO_STATE_MASTER ||
> > > - p_sm->p_subn->sm_state == IB_SMINFO_STATE_DISCOVERING )
> > > - osm_state_mgr_process( &p_sm->state_mgr,
OSM_SIGNAL_SWEEP );
> > > + do {
> > > + signals = 0;
> > > + pthread_mutex_lock(&p_sm->mutex);
> > > + if (p_sm->signal_mask == 0)
> > > + pthread_cond_wait(&p_sm->cond, &p_sm->mutex);
> > > + signals = p_sm->signal_mask;
> > > + p_sm->signal_mask = 0;
> > > + pthread_mutex_unlock(&p_sm->mutex);
> > > + for (i = 0 ; signals ; i++) {
> > > + if (signals&1)
> > > + osm_state_mgr_process( &p_sm->state_mgr, i);
> > > + signals >>= 1;
> > > + }
> > > + } while (p_sm->thread_state == OSM_THREAD_STATE_RUN);
> > >
> > > - /*
> > > - * Wait on the event with a timeout.
> > > - * Sweeps may be iniated "off schedule" by simply
> > > - * signaling the event.
> > > - */
> > > - status = cl_event_wait_on( &p_sm->signal,
> > > - p_sm->p_subn->opt.sweep_interval
*
> > 1000000,
> > > - TRUE );
> > > + OSM_LOG_EXIT( p_sm->p_log );
> > > +}
> > >
> > > - if( status == CL_SUCCESS )
> > > - {
> > > - if( osm_log_is_active( p_sm->p_log, OSM_LOG_DEBUG ) )
> > > - {
> > > - osm_log( p_sm->p_log, OSM_LOG_DEBUG,
> > > - "__osm_sm_sweeper: " "Off schedule sweep
> > signalled\n" );
> > > - }
> > >
> >
+/**********************************************************************
> > > +
> >
**********************************************************************/
> > > +void
> > > +__osm_sm_sweeper_periodic(
> > > + IN void *p_ptr )
> > > +{
> > > + struct timespec times;
> > > + osm_sm_t *const p_sm = ( osm_sm_t * ) p_ptr;
> > > + unsigned i, signals;
> > > + int ret;
> > > +
> > > + OSM_LOG_ENTER( p_sm->p_log, __osm_sm_sweeper_periodic );
> > > +
> > > + do {
> > > + clock_gettime(CLOCK_REALTIME, ×);
> > > + times.tv_sec += p_sm->p_subn->opt.sweep_interval;
> > > + signals = 0;
> > > + pthread_mutex_lock(&p_sm->mutex);
> > > + if (p_sm->signal_mask == 0 &&
> > > + (ret = pthread_cond_timedwait(&p_sm->cond,
&p_sm->mutex,
> > > + ×)) == ETIMEDOUT &&
> > > + ( p_sm->p_subn->sm_state == IB_SMINFO_STATE_MASTER ||
> > > + p_sm->p_subn->sm_state ==
IB_SMINFO_STATE_DISCOVERING
> > )) {
> > > + signals = OSM_SIGNAL_SWEEP;
> > > }
> > > - else
> > > - {
> > > - if( status != CL_TIMEOUT )
> > > - {
> > > - osm_log( p_sm->p_log, OSM_LOG_ERROR,
> > > - "__osm_sm_sweeper: ERR 2E01: "
> > > - "Event wait failed (%s)\n", CL_STATUS_MSG(
> > status ) );
> > > - }
> > > + else {
> > > + signals = p_sm->signal_mask;
> > > + p_sm->signal_mask = 0;
> > > }
> > > - }
> > > + pthread_mutex_unlock(&p_sm->mutex);
> > > + for (i = 0 ; signals ; i++) {
> > > + if (signals&1)
> > > + osm_state_mgr_process( &p_sm->state_mgr, i);
> > > + signals >>= 1;
> > > + }
> > > + } while (p_sm->thread_state == OSM_THREAD_STATE_RUN);
> > >
> > > OSM_LOG_EXIT( p_sm->p_log );
> > > }
> > > @@ -139,7 +153,6 @@ osm_sm_construct(
> > > memset( p_sm, 0, sizeof( *p_sm ) );
> > > p_sm->thread_state = OSM_THREAD_STATE_NONE;
> > > p_sm->sm_trans_id = OSM_SM_INITIAL_TID_VALUE;
> > > - cl_event_construct( &p_sm->signal );
> > > cl_event_construct( &p_sm->subnet_up_event );
> > > cl_thread_construct( &p_sm->sweeper );
> > > osm_req_construct( &p_sm->req );
> > > @@ -158,7 +171,6 @@ osm_sm_construct(
> > > osm_ucast_mgr_construct( &p_sm->ucast_mgr );
> > > osm_link_mgr_construct( &p_sm->link_mgr );
> > > osm_state_mgr_construct( &p_sm->state_mgr );
> > > - osm_state_mgr_ctrl_construct( &p_sm->state_mgr_ctrl );
> > > osm_drop_mgr_construct( &p_sm->drop_mgr );
> > > osm_lft_rcv_construct( &p_sm->lft_rcv );
> > > osm_lft_rcv_ctrl_construct( &p_sm->lft_rcv_ctrl );
> > > @@ -185,24 +197,14 @@ void
> > > osm_sm_shutdown(
> > > IN osm_sm_t * const p_sm )
> > > {
> > > - boolean_t signal_event = FALSE;
> > > -
> > > OSM_LOG_ENTER( p_sm->p_log, osm_sm_shutdown );
> > >
> > > /*
> > > * Signal our threads that we're leaving.
> > > - */
> > > - if( p_sm->thread_state != OSM_THREAD_STATE_NONE )
> > > - signal_event = TRUE;
> > > -
> > > - p_sm->thread_state = OSM_THREAD_STATE_EXIT;
> > > -
> > > - /*
> > > - * Don't trigger unless event has been initialized.
> > > * Destroy the thread before we tear down the other objects.
> > > */
> > > - if( signal_event )
> > > - cl_event_signal( &p_sm->signal );
> > > + p_sm->thread_state = OSM_THREAD_STATE_EXIT;
> > > + osm_sm_signal( p_sm, OSM_SIGNAL_NONE );
> > >
> > > cl_thread_destroy( &p_sm->sweeper );
> > >
> > > @@ -225,7 +227,6 @@ osm_sm_shutdown(
> > > osm_vla_rcv_ctrl_destroy( &p_sm->vla_rcv_ctrl );
> > > osm_pkey_rcv_ctrl_destroy( &p_sm->pkey_rcv_ctrl );
> > > osm_sweep_fail_ctrl_destroy( &p_sm->sweep_fail_ctrl );
> > > - osm_state_mgr_ctrl_destroy( &p_sm->state_mgr_ctrl );
> > >
> > > OSM_LOG_EXIT( p_sm->p_log );
> > > }
> > > @@ -257,12 +258,14 @@ osm_sm_destroy(
> > > osm_state_mgr_destroy( &p_sm->state_mgr );
> > > osm_sm_state_mgr_destroy( &p_sm->sm_state_mgr );
> > > osm_mcast_mgr_destroy( &p_sm->mcast_mgr );
> > > - cl_event_destroy( &p_sm->signal );
> > > cl_event_destroy( &p_sm->subnet_up_event );
> > >
> > > if( p_sm->p_report_buf != NULL )
> > > cl_free( p_sm->p_report_buf );
> > >
> > > + pthread_cond_destroy(&p_sm->cond);
> > > + pthread_mutex_destroy(&p_sm->mutex);
> > > +
> > > osm_log( p_sm->p_log, OSM_LOG_SYS, "Exiting SM\n" ); /* Format
> > Waived */
> > > OSM_LOG_EXIT( p_sm->p_log );
> > > }
> > > @@ -303,14 +306,15 @@ osm_sm_init(
> > > status = IB_INSUFFICIENT_MEMORY;
> > > goto Exit;
> > > }
> > > - status = cl_event_init( &p_sm->signal, FALSE );
> > > - if( status != CL_SUCCESS )
> > > - goto Exit;
> > >
> > > status = cl_event_init( &p_sm->subnet_up_event, FALSE );
> > > if( status != CL_SUCCESS )
> > > goto Exit;
> > >
> > > + p_sm->signal_mask = 0;
> > > + pthread_mutex_init(&p_sm->mutex, NULL);
> > > + pthread_cond_init(&p_sm->cond, NULL);
> > > +
> > > status = osm_sm_mad_ctrl_init( &p_sm->mad_ctrl,
> > > p_sm->p_subn,
> > > p_sm->p_mad_pool,
> > > @@ -416,12 +420,6 @@ osm_sm_init(
> > > if( status != IB_SUCCESS )
> > > goto Exit;
> > >
> > > - status = osm_state_mgr_ctrl_init( &p_sm->state_mgr_ctrl,
> > > - &p_sm->state_mgr,
> > > - p_sm->p_log, p_sm->p_disp );
> > > - if( status != IB_SUCCESS )
> > > - goto Exit;
> > > -
> > > status = osm_drop_mgr_init( &p_sm->drop_mgr,
> > > p_sm->p_subn,
> > > p_sm->p_log, &p_sm->req,
p_sm->p_lock
> > );
> > > @@ -523,16 +521,15 @@ osm_sm_init(
> > >
> > > /*
> > > * Now that the component objects are initialized, start
> > > - * the sweeper thread if the user wants sweeping.
> > > + * the sweeper thread.
> > > */
> > > - if( p_sm->p_subn->opt.sweep_interval )
> > > - {
> > > - p_sm->thread_state = OSM_THREAD_STATE_INIT;
> > > - status = cl_thread_init( &p_sm->sweeper, __osm_sm_sweeper,
> > p_sm,
> > > - "opensm sweeper" );
> > > - if( status != IB_SUCCESS )
> > > - goto Exit;
> > > - }
> > > + p_sm->thread_state = OSM_THREAD_STATE_RUN;
> > > + status = cl_thread_init( &p_sm->sweeper,
> > > + p_sm->p_subn->opt.sweep_interval > 0
?
> > > + __osm_sm_sweeper_periodic :
> > __osm_sm_sweeper,
> > > + p_sm, "opensm sweeper" );
> > > + if( status != IB_SUCCESS )
> > > + goto Exit;
> > >
> > > Exit:
> > > OSM_LOG_EXIT( p_log );
> > > @@ -542,11 +539,26 @@ osm_sm_init(
> > >
> >
/**********************************************************************
> > >
> >
**********************************************************************/
> > > void
> > > +osm_sm_signal(
> > > + IN osm_sm_t* const p_sm,
> > > + IN osm_signal_t signal )
> > > +{
> > > + OSM_LOG_ENTER( p_sm->p_log, osm_sm_signal );
> > > + pthread_mutex_lock(&p_sm->mutex);
> > > + p_sm->signal_mask |= (1 << signal);
> > > + pthread_cond_signal(&p_sm->cond);
> > > + pthread_mutex_unlock(&p_sm->mutex);
> > > + OSM_LOG_EXIT( p_sm->p_log );
> > > +}
> > > +
> > >
> >
+/**********************************************************************
> > > +
> >
**********************************************************************/
> > > +void
> > > osm_sm_sweep(
> > > IN osm_sm_t * const p_sm )
> > > {
> > > OSM_LOG_ENTER( p_sm->p_log, osm_sm_sweep );
> > > - osm_state_mgr_process( &p_sm->state_mgr, OSM_SIGNAL_SWEEP );
> > > + osm_sm_signal( p_sm, OSM_SIGNAL_SWEEP );
> > > OSM_LOG_EXIT( p_sm->p_log );
> > > }
> > >
> > > diff --git a/osm/opensm/osm_sm_mad_ctrl.c
> > b/osm/opensm/osm_sm_mad_ctrl.c
> > > index 9dceef2..1982873 100644
> > > --- a/osm/opensm/osm_sm_mad_ctrl.c
> > > +++ b/osm/opensm/osm_sm_mad_ctrl.c
> > > @@ -81,7 +81,6 @@ __osm_sm_mad_ctrl_retire_trans_mad(
> > > IN osm_madw_t* const p_madw )
> > > {
> > > uint32_t outstanding;
> > > - cl_status_t status;
> > >
> > > OSM_LOG_ENTER( p_ctrl->p_log,
__osm_sm_mad_ctrl_retire_trans_mad );
> > >
> > > @@ -115,31 +114,10 @@ __osm_sm_mad_ctrl_retire_trans_mad(
> > > The wire is clean.
> > > Signal the state manager.
> > > */
> > > - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) )
> > > - {
> > > - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG,
> > > - "__osm_sm_mad_ctrl_retire_trans_mad: "
> > > - "Posting Dispatcher message %s\n",
> > > - osm_get_disp_msg_str( OSM_MSG_NO_SMPS_OUTSTANDING
) );
> > > - }
> > > -
> > > - status = cl_disp_post( p_ctrl->h_disp,
> > > - OSM_MSG_NO_SMPS_OUTSTANDING,
> > > - (void
> > *)OSM_SIGNAL_NO_PENDING_TRANSACTIONS,
> > > - NULL,
> > > - NULL );
> > > -
> > > - if( status != CL_SUCCESS )
> > > - {
> > > - osm_log( p_ctrl->p_log, OSM_LOG_ERROR,
> > > - "__osm_sm_mad_ctrl_retire_trans_mad: ERR 3101: "
> > > - "Dispatcher post message failed (%s)\n",
> > > - CL_STATUS_MSG( status ) );
> > > - goto Exit;
> > > - }
> > > + osm_sm_signal( &p_ctrl->p_subn->p_osm->sm,
> > > + OSM_SIGNAL_NO_PENDING_TRANSACTIONS );
> > > }
> > >
> > > - Exit:
> > > OSM_LOG_EXIT( p_ctrl->p_log );
> > > }
> > > /************/
> > > diff --git a/osm/opensm/osm_sm_state_mgr.c
> > b/osm/opensm/osm_sm_state_mgr.c
> > > index feeda45..2c6da4e 100644
> > > --- a/osm/opensm/osm_sm_state_mgr.c
> > > +++ b/osm/opensm/osm_sm_state_mgr.c
> > > @@ -563,7 +563,7 @@ osm_sm_state_mgr_process(
> > > /*
> > > * Stop the discovering
> > > */
> > > - osm_state_mgr_process( p_sm_mgr->p_state_mgr,
> > > + osm_sm_signal( &p_sm_mgr->p_subn->p_osm->sm,
> > >
> > OSM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED );
> > > break;
> > > case OSM_SM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED_DONE:
> > > @@ -610,7 +610,7 @@ osm_sm_state_mgr_process(
> > > __osm_sm_state_mgr_discovering_msg( p_sm_mgr );
> > > p_sm_mgr->p_subn->sm_state =
IB_SMINFO_STATE_DISCOVERING;
> > > p_sm_mgr->p_subn->coming_out_of_standby = TRUE;
> > > - osm_state_mgr_process( p_sm_mgr->p_state_mgr,
> > OSM_SIGNAL_EXIT_STBY
> > > );
> > > + osm_sm_signal( &p_sm_mgr->p_subn->p_osm->sm,
> > > OSM_SIGNAL_EXIT_STBY );
> > > break;
> > > case OSM_SM_SIGNAL_DISABLE:
> > > /*
> > > @@ -641,7 +641,7 @@ osm_sm_state_mgr_process(
> > > */
> > > p_sm_mgr->p_subn->master_sm_base_lid =
> > p_sm_mgr->p_subn->sm_base_lid;
> > > p_sm_mgr->p_subn->coming_out_of_standby = TRUE;
> > > - osm_state_mgr_process( p_sm_mgr->p_state_mgr,
> > OSM_SIGNAL_EXIT_STBY
> > > );
> > > + osm_sm_signal( &p_sm_mgr->p_subn->p_osm->sm,
> > > OSM_SIGNAL_EXIT_STBY );
> > > break;
> > > case OSM_SM_SIGNAL_ACKNOWLEDGE:
> > > /*
> > > @@ -704,7 +704,7 @@ osm_sm_state_mgr_process(
> > > "Received OSM_SM_SIGNAL_HANDOVER\n" );
> > > p_sm_mgr->p_polling_sm = NULL;
> > > p_sm_mgr->p_subn->force_immediate_heavy_sweep = TRUE;
> > > - osm_state_mgr_process( p_sm_mgr->p_state_mgr,
> > OSM_SIGNAL_SWEEP );
> > > + osm_sm_signal( &p_sm_mgr->p_subn->p_osm->sm,
> > OSM_SIGNAL_SWEEP );
> > > break;
> > > case OSM_SM_SIGNAL_HANDOVER_SENT:
> > > /*
> > > diff --git a/osm/opensm/osm_sminfo_rcv.c
b/osm/opensm/osm_sminfo_rcv.c
> > > index 5914984..4af549b 100644
> > > --- a/osm/opensm/osm_sminfo_rcv.c
> > > +++ b/osm/opensm/osm_sminfo_rcv.c
> > > @@ -425,10 +425,10 @@ __osm_sminfo_rcv_process_set_request(
> > > }
> > >
> > >
> >
/**********************************************************************
> > > - * Return a signal with which to call the osm_state_mgr_process.
> > > + * Return a signal with which to call the osm_sm_signal.
> > > * This is done since we are locked by p_rcv->p_lock in this
> > function,
> > > - * and thus cannot call osm_state_mgr_process (that locks the
> > state_lock).
> > > - * If return OSM_SIGNAL_NONE - do not call osm_state_mgr_process.
> > > + * and thus cannot call osm_sm_signal.
> > > + * If return OSM_SIGNAL_NONE - do not call osm_sm_signal.
> > >
> >
**********************************************************************/
> > > osm_signal_t
> > > __osm_sminfo_rcv_process_get_sm(
> > > @@ -676,7 +676,7 @@ __osm_sminfo_rcv_process_get_response(
> > > /* If process_get_sm_ret_val != OSM_SIGNAL_NONE then we have to
> > signal
> > > * to the state_mgr with that signal. */
> > > if (process_get_sm_ret_val != OSM_SIGNAL_NONE)
> > > - osm_state_mgr_process( p_rcv->p_state_mgr,
> > > + osm_sm_signal( &p_rcv->p_subn->p_osm->sm,
> > > process_get_sm_ret_val );
> > > OSM_LOG_EXIT( p_rcv->p_log );
> > > }
> > > diff --git a/osm/opensm/osm_state_mgr.c
b/osm/opensm/osm_state_mgr.c
> > > index 724b2b7..ff1c65c 100644
> > > --- a/osm/opensm/osm_state_mgr.c
> > > +++ b/osm/opensm/osm_state_mgr.c
> > > @@ -83,7 +83,6 @@ osm_state_mgr_construct(
> > > IN osm_state_mgr_t * const p_mgr )
> > > {
> > > memset( p_mgr, 0, sizeof( *p_mgr ) );
> > > - cl_spinlock_construct( &p_mgr->state_lock );
> > > cl_spinlock_construct( &p_mgr->idle_lock );
> > > p_mgr->state = OSM_SM_STATE_INIT;
> > > }
> > > @@ -99,7 +98,6 @@ osm_state_mgr_destroy(
> > > OSM_LOG_ENTER( p_mgr->p_log, osm_state_mgr_destroy );
> > >
> > > /* destroy the locks */
> > > - cl_spinlock_destroy( &p_mgr->state_lock );
> > > cl_spinlock_destroy( &p_mgr->idle_lock );
> > >
> > > OSM_LOG_EXIT( p_mgr->p_log );
> > > @@ -162,14 +160,6 @@ osm_state_mgr_init(
> > > p_mgr->state_step_mode = OSM_STATE_STEP_CONTINUOUS;
> > > p_mgr->next_stage_signal = OSM_SIGNAL_NONE;
> > >
> > > - status = cl_spinlock_init( &p_mgr->state_lock );
> > > - if( status != CL_SUCCESS )
> > > - {
> > > - osm_log( p_mgr->p_log, OSM_LOG_ERROR,
> > > - "osm_state_mgr_init: ERR 3301: "
> > > - "Spinlock init failed (%s)\n", CL_STATUS_MSG(
status )
> > );
> > > - }
> > > -
> > > cl_qlist_init( &p_mgr->idle_time_list );
> > >
> > > status = cl_spinlock_init( &p_mgr->idle_lock );
> > > @@ -1897,16 +1887,6 @@ osm_state_mgr_process(
> > > if( osm_exit_flag )
> > > signal = OSM_SIGNAL_NONE;
> > >
> > > - /*
> > > - * The state lock prevents many race conditions from screwing
> > > - * up the state transition process. For example, if an
function
> > > - * puts transactions on the wire, the state lock guarantees
this
> > > - * loop will see the return code ("DONE PENDING") of the
function
> > > - * before the "NO OUTSTANDING TRANSACTIONS" signal is
> > asynchronously
> > > - * received.
> > > - */
> > > - cl_spinlock_acquire( &p_mgr->state_lock );
> > > -
> > > while( signal != OSM_SIGNAL_NONE )
> > > {
> > > if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) )
> > > @@ -2957,8 +2937,6 @@ osm_state_mgr_process(
> > > p_mgr->state_step_mode = OSM_STATE_STEP_BREAK;
> > > }
> > >
> > > - cl_spinlock_release( &p_mgr->state_lock );
> > > -
> > > OSM_LOG_EXIT( p_mgr->p_log );
> > > }
> > >
> > > @@ -2994,7 +2972,7 @@ osm_state_mgr_process_idle(
> > > cl_qlist_insert_tail( &p_mgr->idle_time_list,
> > &p_idle_item->list_item );
> > > cl_spinlock_release( &p_mgr->idle_lock );
> > >
> > > - osm_state_mgr_process( p_mgr,
> OSM_SIGNAL_IDLE_TIME_PROCESS_REQUEST
> > > );
> > > + osm_sm_signal( &p_mgr->p_subn->p_osm->sm,
> > > OSM_SIGNAL_IDLE_TIME_PROCESS_REQUEST );
> > >
> > > OSM_LOG_EXIT( p_mgr->p_log );
> > >
> > > diff --git a/osm/opensm/osm_state_mgr_ctrl.c
> > b/osm/opensm/osm_state_mgr_ctrl.c
> > > deleted file mode 100644
> > > index 0bde333..0000000
> > > --- a/osm/opensm/osm_state_mgr_ctrl.c
> > > +++ /dev/null
> > > @@ -1,132 +0,0 @@
> > > -/*
> > > - * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved.
> > > - * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights
> > reserved.
> > > - * Copyright (c) 1996-2003 Intel Corporation. All rights
reserved.
> > > - *
> > > - * This software is available to you under a choice of one of two
> > > - * licenses. You may choose to be licensed under the terms of
the
> > GNU
> > > - * General Public License (GPL) Version 2, available from the
file
> > > - * COPYING in the main directory of this source tree, or the
> > > - * OpenIB.org BSD license below:
> > > - *
> > > - * Redistribution and use in source and binary forms, with or
> > > - * without modification, are permitted provided that the
> > following
> > > - * conditions are met:
> > > - *
> > > - * - Redistributions of source code must retain the above
> > > - * copyright notice, this list of conditions and the
following
> > > - * disclaimer.
> > > - *
> > > - * - Redistributions in binary form must reproduce the above
> > > - * copyright notice, this list of conditions and the
following
> > > - * disclaimer in the documentation and/or other materials
> > > - * provided with the distribution.
> > > - *
> > > - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
> KIND,
> > > - * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
> > > WARRANTIES OF
> > > - * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
> > > - * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
> COPYRIGHT
> > > HOLDERS
> > > - * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
> IN
> > > AN
> > > - * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF
> OR
> > > IN
> > > - * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> IN
> > > THE
> > > - * SOFTWARE.
> > > - *
> > > - * $Id$
> > > - */
> > > -
> > > -
> > > -/*
> > > - * Abstract:
> > > - * Implementation of osm_state_mgr_ctrl_t.
> > > - * This object represents the State Manager Controller object.
> > > - * This object is part of the opensm family of objects.
> > > - *
> > > - * Environment:
> > > - * Linux User Mode
> > > - *
> > > - * $Revision: 1.5 $
> > > - */
> > > -
> > > -/*
> > > - Next available error code: 0x1601
> > > -*/
> > > -
> > > -#if HAVE_CONFIG_H
> > > -# include <config.h>
> > > -#endif /* HAVE_CONFIG_H */
> > > -
> > > -#include <string.h>
> > > -#include <opensm/osm_state_mgr_ctrl.h>
> > > -#include <opensm/osm_msgdef.h>
> > > -
> > >
> >
-/**********************************************************************
> > > -
> >
**********************************************************************/
> > > -void
> > > -__osm_state_mgr_ctrl_disp_callback(
> > > - IN void *context,
> > > - IN void *p_data )
> > > -{
> > > - /* ignore return status when invoked via the dispatcher */
> > > - osm_state_mgr_process(
((osm_state_mgr_ctrl_t*)context)->p_mgr,
> > > - (osm_signal_t)(p_data) );
> > > -}
> > > -
> > >
> >
-/**********************************************************************
> > > -
> >
**********************************************************************/
> > > -void
> > > -osm_state_mgr_ctrl_construct(
> > > - IN osm_state_mgr_ctrl_t* const p_ctrl )
> > > -{
> > > - memset( p_ctrl, 0, sizeof(*p_ctrl) );
> > > - p_ctrl->h_disp = CL_DISP_INVALID_HANDLE;
> > > -}
> > > -
> > >
> >
-/**********************************************************************
> > > -
> >
**********************************************************************/
> > > -void
> > > -osm_state_mgr_ctrl_destroy(
> > > - IN osm_state_mgr_ctrl_t* const p_ctrl )
> > > -{
> > > - CL_ASSERT( p_ctrl );
> > > - cl_disp_unregister( p_ctrl->h_disp );
> > > -}
> > > -
> > >
> >
-/**********************************************************************
> > > -
> >
**********************************************************************/
> > > -ib_api_status_t
> > > -osm_state_mgr_ctrl_init(
> > > - IN osm_state_mgr_ctrl_t* const p_ctrl,
> > > - IN osm_state_mgr_t* const p_mgr,
> > > - IN osm_log_t* const p_log,
> > > - IN cl_dispatcher_t* const p_disp )
> > > -{
> > > - ib_api_status_t status = IB_SUCCESS;
> > > -
> > > - OSM_LOG_ENTER( p_log, osm_state_mgr_ctrl_init );
> > > -
> > > - osm_state_mgr_ctrl_construct( p_ctrl );
> > > - p_ctrl->p_log = p_log;
> > > -
> > > - p_ctrl->p_mgr = p_mgr;
> > > - p_ctrl->p_disp = p_disp;
> > > -
> > > - p_ctrl->h_disp = cl_disp_register(
> > > - p_disp,
> > > - OSM_MSG_NO_SMPS_OUTSTANDING,
> > > - __osm_state_mgr_ctrl_disp_callback,
> > > - p_ctrl );
> > > -
> > > - if( p_ctrl->h_disp == CL_DISP_INVALID_HANDLE )
> > > - {
> > > - osm_log( p_log, OSM_LOG_ERROR,
> > > - "osm_state_mgr_ctrl_init: ERR 3401: "
> > > - "Dispatcher registration failed\n" );
> > > - status = IB_INSUFFICIENT_RESOURCES;
> > > - goto Exit;
> > > - }
> > > -
> > > - Exit:
> > > - OSM_LOG_EXIT( p_log );
> > > - return( status );
> > > -}
> > > -
> > > diff --git a/osm/opensm/osm_sw_info_rcv.c
> > b/osm/opensm/osm_sw_info_rcv.c
> > > index 6bbd73a..61aff27 100644
> > > --- a/osm/opensm/osm_sw_info_rcv.c
> > > +++ b/osm/opensm/osm_sw_info_rcv.c
> > > @@ -60,6 +60,7 @@ #include <opensm/osm_sw_info_rcv.h>
> > > #include <opensm/osm_log.h>
> > > #include <opensm/osm_switch.h>
> > > #include <opensm/osm_subnet.h>
> > > +#include <opensm/osm_opensm.h>
> > > #include <opensm/osm_helper.h>
> > >
> > >
> >
/**********************************************************************
> > > @@ -673,7 +674,7 @@ osm_si_rcv_process(
> > > if (__osm_si_rcv_process_existing( p_rcv, p_node, p_sw,
p_madw
> > ))
> > > {
> > > CL_PLOCK_RELEASE( p_rcv->p_lock );
> > > - osm_state_mgr_process( p_rcv->p_state_mgr,
> > > + osm_sm_signal( &p_rcv->p_subn->p_osm->sm,
> > > OSM_SIGNAL_CHANGE_DETECTED );
> > > goto Exit;
> > > }
> > > diff --git a/osm/opensm/osm_sweep_fail_ctrl.c
> > b/osm/opensm/osm_sweep_fail_ctrl.c
> > > index e27a540..9e41ec7 100644
> > > --- a/osm/opensm/osm_sweep_fail_ctrl.c
> > > +++ b/osm/opensm/osm_sweep_fail_ctrl.c
> > > @@ -52,6 +52,8 @@ #endif /* HAVE_CONFIG_H */
> > > #include <string.h>
> > > #include <opensm/osm_sweep_fail_ctrl.h>
> > > #include <opensm/osm_msgdef.h>
> > > +#include <opensm/osm_sm.h>
> > > +#include <opensm/osm_opensm.h>
> > >
> > >
> >
/**********************************************************************
> > >
> >
**********************************************************************/
> > > @@ -68,7 +70,7 @@ __osm_sweep_fail_ctrl_disp_callback(
> > > /*
> > > Notify the state manager that we had a light sweep failure.
> > > */
> > > - osm_state_mgr_process( p_ctrl->p_state_mgr,
> > > + osm_sm_signal( &p_ctrl->p_state_mgr->p_subn->p_osm->sm,
> > > OSM_SIGNAL_LIGHT_SWEEP_FAIL );
> > >
> > > OSM_LOG_EXIT( p_ctrl->p_log );
> > > diff --git a/osm/opensm/osm_trap_rcv.c b/osm/opensm/osm_trap_rcv.c
> > > index 9865f53..fb32ce9 100644
> > > --- a/osm/opensm/osm_trap_rcv.c
> > > +++ b/osm/opensm/osm_trap_rcv.c
> > > @@ -589,8 +589,7 @@ __osm_trap_rcv_process_request(
> > >
> > > p_rcv->p_subn->force_immediate_heavy_sweep = TRUE;
> > > }
> > > - osm_state_mgr_process( p_rcv->p_state_mgr,
> > > - OSM_SIGNAL_SWEEP );
> > > + osm_sm_signal( &p_rcv->p_subn->p_osm->sm, OSM_SIGNAL_SWEEP );
> > > }
> > >
> > > /* If we reached here due to trap 129/130/131 - do not need to
do
> > > diff --git a/osm/opensm/osm_vl15intf.c b/osm/opensm/osm_vl15intf.c
> > > index 68f17c5..c3adb6e 100644
> > > --- a/osm/opensm/osm_vl15intf.c
> > > +++ b/osm/opensm/osm_vl15intf.c
> > > @@ -62,6 +62,7 @@ #include <opensm/osm_madw.h>
> > > #include <vendor/osm_vendor_api.h>
> > > #include <opensm/osm_log.h>
> > > #include <opensm/osm_helper.h>
> > > +#include <opensm/osm_opensm.h>
> > > #include <complib/cl_thread.h>
> > > #include <signal.h>
> > >
> > > @@ -156,7 +157,6 @@ __osm_vl15_poller(
> > > if( status != IB_SUCCESS )
> > > {
> > > uint32_t outstanding;
> > > - cl_status_t cl_status;
> > >
> > > osm_log( p_vl->p_log, OSM_LOG_ERROR,
> > > "__osm_vl15_poller: ERR 3E03: "
> > > @@ -202,27 +202,8 @@ __osm_vl15_poller(
> > > The wire is clean.
> > > Signal the state manager.
> > > */
> > > - if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) )
> > > - {
> > > - osm_log( p_vl->p_log, OSM_LOG_DEBUG,
> > > - "__osm_vl15_poller: "
> > > - "Posting Dispatcher message %s\n",
> > > - osm_get_disp_msg_str(
> > OSM_MSG_NO_SMPS_OUTSTANDING ) );
> > > - }
> > > -
> > > - cl_status = cl_disp_post( p_vl->h_disp,
> > > -
OSM_MSG_NO_SMPS_OUTSTANDING,
> > > - (void
> > *)OSM_SIGNAL_NO_PENDING_TRANSACTIONS,
> > > - NULL,
> > > - NULL );
> > > -
> > > - if( cl_status != CL_SUCCESS )
> > > - {
> > > - osm_log( p_vl->p_log, OSM_LOG_ERROR,
> > > - "__osm_vl15_poller: ERR 3E06: "
> > > - "Dispatcher post message failed (%s)\n",
> > > - CL_STATUS_MSG( cl_status ) );
> > > - }
> > > + osm_sm_signal( &p_vl->p_subn->p_osm->sm,
> > > + OSM_SIGNAL_NO_PENDING_TRANSACTIONS );
> > > }
> > > }
> > > }
More information about the general
mailing list