<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.3243" name=GENERATOR></HEAD>
<BODY>
<DIV><SPAN class=406463711-02112008><FONT face=Arial color=#0000ff
size=2>Applied in 1721,1722.</FONT></SPAN></DIV>
<DIV><SPAN class=406463711-02112008><FONT face=Arial color=#0000ff size=2>Thank
you, Alex.</FONT></SPAN></DIV><BR>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> ofw-bounces@lists.openfabrics.org
[mailto:ofw-bounces@lists.openfabrics.org] <B>On Behalf Of </B>Leonid
Keller<BR><B>Sent:</B> Wednesday, October 29, 2008 2:57 PM<BR><B>To:</B> Alex
Naslednikov; ofw@lists.openfabrics.org; Smith, Stan<BR><B>Subject:</B> RE:
[ofw] [patch] [ibbus] Hibernation bugfix 1/2<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=998335012-29102008>>
<FONT color=#000000>+ //Setting h_ca to be NULL forces IPoIB to start
only after h_ca will be recreated</FONT></SPAN></FONT></DIV>
<DIV><FONT face=Arial size=2><SPAN
class=998335012-29102008></SPAN></FONT> </DIV>
<DIV><FONT face=Arial size=2><SPAN class=998335012-29102008>I'd suggest a more
detailed comment</SPAN></FONT></DIV>
<DIV><FONT face=Arial size=2><SPAN class=998335012-29102008>// Setting
h_ca to be NULL forces IPoIB to start only after re-acquiring new CA
object</SPAN></FONT></DIV>
<DIV><FONT face=Arial size=2><SPAN class=998335012-29102008>// The latter
happens in __port_was_hibernated or port_mgr_port_add functions
</SPAN></FONT></DIV>
<DIV><FONT face=Arial size=2><SPAN class=998335012-29102008>// after arriving
IB_PNP_PORT_ADD event from IBAL</SPAN></FONT></DIV><BR>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> ofw-bounces@lists.openfabrics.org
[mailto:ofw-bounces@lists.openfabrics.org] <B>On Behalf Of </B>Alex
Naslednikov<BR><B>Sent:</B> Wednesday, October 29, 2008 11:14
AM<BR><B>To:</B> ofw@lists.openfabrics.org; Smith, Stan<BR><B>Subject:</B>
[ofw] [patch] [ibbus] Hibernation bugfix 1/2<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial
size=2>Explanations:</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>Hibernation
mechanism was broken after filter driver check-ins. </FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>The current
situation is that when restoring from hibernation state, IPoIB adapters
can't be recreated.</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>That is because
the following race:</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>1. IPoIB sends
the following IRP during the initialization :
IRP_MN_QUERY_INTERFACE</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>2. This request
is handled in port_query_interface that calls cl_fwd_query_ifc with the
following argument:</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial
size=2>p_ext->h_ca->obj.p_ci_ca->verbs.p_hca_dev</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>We found that
p_hca_dev struct is almost always NULL or contains invalid pointer after
restoring the system from the hibernation</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>And that is
because h_ca object still was uninitialized prior to this Query Interface
request?</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial
size=2>Why?</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>3. We have a
following check to avoid this race (i.e. avoid h_ca to be created before
IPoIB will query the interface):</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>In
</FONT></SPAN><SPAN class=765405108-29102008><FONT face=Arial
size=2>_HibernateUpWorkItem(<BR> IN DEVICE_OBJECT* p_dev_obj,<BR> IN void* context
)<BR>{<BR> .....</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2> while
(!p_ext->h_ca) {<BR> BUS_TRACE( BUS_DBG_PNP, ("Waiting for the
end of HCA registration ... \n"));<BR> cl_thread_suspend( 200
); /* suspend for 200 ms */<BR> }</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial
size=2></FONT></SPAN> </DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>4. According to
the above code, somebody should clear h_ca field after dereferencing
it.</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>This code was
exist and occasionally removed thereafter</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial
size=2></FONT></SPAN> </DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>5. Below is the
patch:</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>Index:
core/bus/kernel/bus_port_mgr.c<BR>===================================================================</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>Fixing a bug in
hibernation mechanism</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial size=2>Signed-off by:
leonid at mellanox.com</FONT></SPAN></DIV>
<DIV><SPAN class=765405108-29102008><FONT face=Arial
size=2>
xalex at mellanox.com<BR>--- core/bus/kernel/bus_port_mgr.c (revision
3373)<BR>+++ core/bus/kernel/bus_port_mgr.c (working copy)<BR>@@
-1258,6 +1258,8
@@<BR> <BR> hca_deref:<BR> deref_al_obj(
&p_ext->h_ca->obj );<BR>+ //Setting h_ca to be NULL forces
IPoIB to start only after h_ca will be recreated<BR>+ p_ext->h_ca =
NULL;<BR> cl_mutex_release( &gp_port_mgr->pdo_mutex
);<BR> <BR> BUS_EXIT( BUS_DBG_PNP );<BR>@@ -1849,8 +1851,7
@@<BR> <BR> p_ext =
p_dev_obj->DeviceExtension;<BR> if
(p_ext->pdo.b_hibernating) {<BR>- // Can't continue
within hibernation stage<BR>- return
STATUS_UNSUCCESSFUL;<BR>+ BUS_TRACE( BUS_DBG_PNP, ("Restoring
from the hibernation: Device query received.\n")
);<BR> }<BR> BUS_TRACE( BUS_DBG_PNP, ("Query i/f for
%s: PDO %p (=%p),ext %p, present %d, missing %d
.\n",<BR> p_ext->pdo.cl_ext.vfptr_pnp_po->identity,
p_ext->pdo.cl_ext.p_self_do, <BR></DIV></FONT></SPAN>
<DIV><SPAN class=765405108-29102008><FONT face=Arial
size=2></FONT></SPAN> </DIV></BLOCKQUOTE></BLOCKQUOTE></BODY></HTML>