<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:w =
"urn:schemas-microsoft-com:office:word" xmlns:o =
"urn:schemas-microsoft-com:office:office"><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<STYLE>@font-face {
font-family: Tahoma;
}
@page Section1 {size: 8.5in 11.0in; margin: 1.0in 1.25in 1.0in 1.25in; }
P.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
LI.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
DIV.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
A:link {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline
}
SPAN.EmailStyle17 {
FONT-WEIGHT: normal; COLOR: windowtext; FONT-STYLE: normal; FONT-FAMILY: Tahoma; TEXT-DECORATION: none; mso-style-type: personal
}
SPAN.EmailStyle18 {
FONT-WEIGHT: normal; COLOR: blue; FONT-STYLE: normal; FONT-FAMILY: Tahoma; TEXT-DECORATION: none; mso-style-type: personal-reply
}
DIV.Section1 {
page: Section1
}
</STYLE>
<META content="MSHTML 6.00.2900.3243" name=GENERATOR></HEAD>
<BODY lang=EN-US vLink=purple link=blue>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=979034418-31072008>Find
attached a fix, that will maybe solve the problem.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=979034418-31072008>Maybe,
because i haven't time to check it and this work week (down here) is
ended.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=979034418-31072008>I'll
check the patch next week.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=979034418-31072008></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=979034418-31072008>About
the patch.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=979034418-31072008></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=979034418-31072008>First
of all, special thanks to Anatoly for the right pointing to 1435
patch.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=979034418-31072008></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=979034418-31072008>To
recall, 1435 patch has improved event notification mechanism for cq, qp and
srq objects.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=979034418-31072008>I
found one problem in the patch, which repeats itself for all three objects and
for both drivers: new event handlers get the old (and wrong) context
values.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=979034418-31072008>The
new (and right) context values are nor used. As a result, IBAL
callbacks are called with wrong handle parameter, which ends up with
crash.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=979034418-31072008></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT><FONT face=Arial color=#0000ff
size=2></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2>Index:
hw/mlx4/kernel/bus/inc/ib_verbs.h<BR>===================================================================<BR>---
hw/mlx4/kernel/bus/inc/ib_verbs.h (revision 1452)<BR>+++
hw/mlx4/kernel/bus/inc/ib_verbs.h (working copy)<BR>@@ -742,7 +742,6
@@<BR> void
*
cq_context;<BR> int
cqe;<BR> atomic_t
usecnt; /* count number of work queues */<BR>- struct
ib_cq_ex x;<BR> };<BR> <BR> struct ib_srq
{<BR>@@ -752,7 +751,6
@@<BR> void
(*event_handler)(ib_event_rec_t
*);<BR> void
*srq_context;<BR> atomic_t usecnt;<BR>- struct
ib_srq_ex
x;<BR> };<BR> <BR> struct ib_qp {<BR>@@ -766,7 +764,6
@@<BR> void
*qp_context;<BR> u32 qp_num;<BR> enum
ib_qp_type qp_type;<BR>- struct
ib_qp_ex
x;<BR> };<BR> <BR> struct ib_mr {<BR>Index:
hw/mlx4/kernel/bus/inc/ib_verbs_ex.h<BR>===================================================================<BR>---
hw/mlx4/kernel/bus/inc/ib_verbs_ex.h (revision 1452)<BR>+++
hw/mlx4/kernel/bus/inc/ib_verbs_ex.h (working copy)<BR>@@ -73,24 +73,6
@@<BR> int
fw_if_open;<BR> };<BR> <BR>-/* extension for
ib_cq */<BR>-struct ib_cq_ex <BR>-{<BR>- void
* ctx; /* IBAL CQ context
*/<BR>-};<BR>-<BR>-/* extension for ib_qp */<BR>-struct ib_qp_ex
<BR>-{<BR>- void * ctx; /* IBAL QP
context */<BR>-};<BR>-<BR>-/* extension for ib_srq */<BR>-struct ib_srq_ex
<BR>-{<BR>- void * ctx; /* IBAL SRQ
context */<BR>-};<BR>-<BR> /* extension for ib_event */<BR> struct
ib_event_ex <BR> {<BR>Index:
hw/mlx4/kernel/hca/cq.c<BR>===================================================================<BR>---
hw/mlx4/kernel/hca/cq.c (revision 1452)<BR>+++
hw/mlx4/kernel/hca/cq.c (working copy)<BR>@@ -89,7 +89,7
@@<BR> // allocate cq <BR> p_ib_cq =
ibv_create_cq(p_ibdev, <BR> cq_comp_handler,
event_handler,<BR>- p_hca, *p_size, p_uctx, p_umv_buf
);<BR>+ (void*)cq_context, *p_size, p_uctx, p_umv_buf
);<BR> if (IS_ERR(p_ib_cq)) {<BR> err =
PTR_ERR(p_ib_cq);<BR> HCA_PRINT (TRACE_LEVEL_ERROR ,HCA_DBG_CQ,
("ibv_create_cq failed (%d)\n", err));<BR>@@ -97,9 +97,6
@@<BR> goto
err_create_cq;<BR> }<BR> <BR>- // fill the
object<BR>- p_ib_cq->x.ctx =
(void*)cq_context;<BR>- <BR> // return the
result<BR> *p_size = p_ib_cq->cqe;<BR> <BR>Index:
hw/mlx4/kernel/hca/qp.c<BR>===================================================================<BR>---
hw/mlx4/kernel/hca/qp.c (revision 1452)<BR>+++
hw/mlx4/kernel/hca/qp.c (working copy)<BR>@@ -100,8 +100,6
@@<BR> struct ib_qp_init_attr qp_init_attr;<BR> struct
ib_ucontext *p_uctx = NULL;<BR> struct ib_pd *p_ib_pd = (struct ib_pd
*)h_pd;<BR>- struct ib_device *p_ib_dev =
p_ib_pd->device;<BR>- mlnx_hca_t *p_hca =
ibdev2hca(p_ib_dev);<BR> struct ibv_create_qp *p_req =
NULL;<BR> <BR> HCA_ENTER(HCA_DBG_QP);<BR>@@ -121,7 +119,7
@@<BR> // prepare the
parameters<BR> RtlZeroMemory(&qp_init_attr,
sizeof(qp_init_attr));<BR> qp_init_attr.event_handler =
event_handler;<BR>- qp_init_attr.qp_context =
p_hca;<BR>+ qp_init_attr.qp_context =
(void*)qp_uctx;<BR> qp_init_attr.send_cq = (struct ib_cq
*)p_create_attr->h_sq_cq;<BR> qp_init_attr.recv_cq = (struct ib_cq
*)p_create_attr->h_rq_cq;<BR> qp_init_attr.srq = (struct ib_srq
*)p_create_attr->h_srq;<BR>@@ -153,9 +151,6 @@<BR> goto
err_create_qp;<BR> }<BR> <BR>- // fill the
object<BR>- p_ib_qp->x.ctx = (void*)qp_uctx;<BR>-<BR> //
Query QP to obtain requested attributes<BR> if (p_qp_attr)
{<BR> status = mlnx_query_qp((ib_qp_handle_t)p_ib_qp,
p_qp_attr, p_umv_buf);<BR>Index:
hw/mlx4/kernel/hca/srq.c<BR>===================================================================<BR>---
hw/mlx4/kernel/hca/srq.c (revision 1452)<BR>+++
hw/mlx4/kernel/hca/srq.c (working copy)<BR>@@ -55,8 +55,6
@@<BR> struct ib_srq_init_attr srq_init_attr;<BR> struct
ib_ucontext *p_uctx = NULL;<BR> struct ib_pd *p_ib_pd = (struct ib_pd
*)h_pd;<BR>- struct ib_device *p_ib_dev =
p_ib_pd->device;<BR>- mlnx_hca_t *p_hca =
ibdev2hca(p_ib_dev);<BR> <BR> HCA_ENTER(HCA_DBG_SRQ);<BR> <BR>@@
-75,7 +73,7 @@<BR> // prepare the
parameters<BR> RtlZeroMemory(&srq_init_attr,
sizeof(srq_init_attr));<BR> srq_init_attr.event_handler =
event_handler;<BR>- srq_init_attr.srq_context =
p_hca;<BR>+ srq_init_attr.srq_context =
(void*)srq_context;<BR> srq_init_attr.attr.max_wr =
p_srq_attr->max_wr;<BR> srq_init_attr.attr.max_sge =
p_srq_attr->max_sge;<BR> srq_init_attr.attr.srq_limit =
p_srq_attr->srq_limit;<BR>@@ -88,7 +86,6 @@<BR> status =
errno_to_iberr(err);<BR> goto
err_create_srq;<BR> }<BR>- p_ib_srq->x.ctx =
(void*)srq_context;<BR> <BR> // return the
result<BR> if (ph_srq) *ph_srq = (ib_srq_handle_t)p_ib_srq;<BR>Index:
hw/mthca/kernel/hca_verbs.c<BR>===================================================================<BR>---
hw/mthca/kernel/hca_verbs.c (revision 1452)<BR>+++
hw/mthca/kernel/hca_verbs.c (working copy)<BR>@@ -870,12 +870,10
@@<BR> int err;<BR> ib_api_status_t
status;<BR> struct ib_srq *ib_srq_p;<BR>- struct mthca_srq
*srq_p;<BR> struct ib_srq_init_attr
srq_init_attr;<BR> struct ib_ucontext *p_context =
NULL;<BR> struct ib_pd *ib_pd_p = (struct ib_pd
*)h_pd;<BR> struct ib_device *ib_dev =
ib_pd_p->device;<BR>- mlnx_hob_t *hob_p =
HOB_FROM_IBDEV(ib_dev);<BR> <BR> HCA_ENTER(HCA_DBG_SRQ);<BR> <BR>@@
-894,7 +892,7 @@<BR> // prepare the
parameters<BR> RtlZeroMemory(&srq_init_attr,
sizeof(srq_init_attr));<BR> srq_init_attr.event_handler =
event_handler;<BR>- srq_init_attr.srq_context =
hob_p;<BR>+ srq_init_attr.srq_context =
(void*)srq_context;<BR> srq_init_attr.attr =
*p_srq_attr;<BR> <BR> // allocate srq <BR>@@ -906,12 +904,8
@@<BR> goto
err_create_srq;<BR> }<BR> <BR>- // fill the
object<BR>- srq_p = (struct mthca_srq
*)ib_srq_p;<BR>- srq_p->srq_context =
(void*)srq_context;<BR>- <BR> // return the result<BR>- if
(ph_srq) *ph_srq = (ib_srq_handle_t)srq_p;<BR>+ if (ph_srq) *ph_srq =
(ib_srq_handle_t)ib_srq_p;<BR> <BR> status =
IB_SUCCESS;<BR> <BR>@@ -1044,7 +1038,6 @@<BR> struct
ib_ucontext *p_context = NULL;<BR> struct ib_pd *ib_pd_p = (struct
ib_pd *)h_pd;<BR> struct ib_device *ib_dev =
ib_pd_p->device;<BR>- mlnx_hob_t *hob_p =
HOB_FROM_IBDEV(ib_dev);<BR> <BR> HCA_ENTER(HCA_DBG_QP);<BR> <BR>@@
-1063,7 +1056,7 @@<BR> RtlZeroMemory(&qp_init_attr,
sizeof(qp_init_attr));<BR> qp_init_attr.qp_type =
p_create_attr->qp_type;<BR> qp_init_attr.event_handler =
event_handler;<BR>- qp_init_attr.qp_context =
hob_p;<BR>+ qp_init_attr.qp_context =
(void*)qp_context;<BR> qp_init_attr.recv_cq = (struct ib_cq
*)p_create_attr->h_rq_cq;<BR> qp_init_attr.send_cq = (struct ib_cq
*)p_create_attr->h_sq_cq;<BR> qp_init_attr.srq = (struct ib_srq
*)p_create_attr->h_srq;<BR>@@ -1087,7 +1080,6 @@<BR> <BR> //
fill the object<BR> qp_p = (struct mthca_qp
*)ib_qp_p;<BR>- qp_p->qp_context =
(void*)qp_context;<BR> qp_p->qp_init_attr =
qp_init_attr;<BR> <BR> // Query QP to obtain requested
attributes<BR>@@ -1401,7 +1393,6 @@<BR> int
err;<BR> ib_api_status_t status;<BR> struct ib_cq
*ib_cq_p;<BR>- struct mthca_cq
*cq_p;<BR> mlnx_hob_t *hob_p;<BR> struct
ib_device *ib_dev;<BR> struct ib_ucontext *p_context;<BR>@@ -1437,7
+1428,7 @@<BR> // allocate cq <BR> ib_cq_p =
ibv_create_cq(ib_dev, <BR> cq_comp_handler,
event_handler,<BR>- hob_p, *p_size, p_context, p_umv_buf
);<BR>+ (void*)cq_context, *p_size, p_context, p_umv_buf
);<BR> if (IS_ERR(ib_cq_p)) {<BR> err =
PTR_ERR(ib_cq_p);<BR> HCA_PRINT (TRACE_LEVEL_ERROR ,HCA_DBG_CQ,
("ibv_create_cq failed (%d)\n", err));<BR>@@ -1445,15 +1436,11
@@<BR> goto
err_create_cq;<BR> }<BR> <BR>- // fill the
object<BR>- cq_p = (struct mthca_cq
*)ib_cq_p;<BR>- cq_p->cq_context =
(void*)cq_context;<BR>- <BR> // return the
result<BR> // *p_size = *p_size; // return the same
value<BR> *p_size = ib_cq_p->cqe;<BR> <BR>- if (ph_cq)
*ph_cq = (ib_cq_handle_t)cq_p;<BR>+ if (ph_cq) *ph_cq =
(ib_cq_handle_t)ib_cq_p;<BR> <BR> status =
IB_SUCCESS;<BR> <BR>Index:
hw/mthca/kernel/mthca_cq.c<BR>===================================================================<BR>---
hw/mthca/kernel/mthca_cq.c (revision 1452)<BR>+++
hw/mthca/kernel/mthca_cq.c (working copy)<BR>@@ -237,7 +237,7
@@<BR> ++cq->arm_sn;<BR> }<BR> <BR>- cq->ibcq.comp_handler(cq->cq_context);<BR>+ cq->ibcq.comp_handler(cq->ibcq.cq_context);<BR> }<BR> <BR> void
mthca_cq_event(struct mthca_dev *dev, u32 cqn,<BR>Index:
hw/mthca/kernel/mthca_provider.h<BR>===================================================================<BR>---
hw/mthca/kernel/mthca_provider.h (revision 1452)<BR>+++
hw/mthca/kernel/mthca_provider.h (working copy)<BR>@@ -189,7 +189,6
@@<BR> <BR> struct mthca_cq {<BR> struct
ib_cq
ibcq;<BR>- void *cq_context; //
leo: for IBAL
shim<BR> spinlock_t
lock;<BR> atomic_t
refcount;<BR> int
cqn;<BR>@@ -234,7 +233,6
@@<BR> <BR> wait_queue_head_t wait;<BR> KMUTEX mutex;<BR>- void *srq_context; <BR> };<BR> <BR> struct
mthca_wq {<BR>@@ -254,7 +252,6 @@<BR> <BR> struct mthca_qp
{<BR> struct
ib_qp
ibqp;<BR>- void *qp_context; //
leo: for IBAL shim<BR> //TODO: added just because absense of
ibv_query_qp<BR> // thereafter it may be worth to be replaced by
struct ib_qp_attr qp_attr;<BR> struct ib_qp_init_attr
qp_init_attr; // leo: for query_qp<BR></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT> </DIV>
<DIV><BR></DIV>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> ofw-bounces@lists.openfabrics.org
[mailto:ofw-bounces@lists.openfabrics.org] <B>On Behalf Of </B>Anatoly
Greenblatt<BR><B>Sent:</B> Wednesday, July 30, 2008 2:35 PM<BR><B>To:</B>
ofw@lists.openfabrics.org<BR><B>Subject:</B> [ofw] ConnectX functionality is
completely broken<BR><B>Importance:</B> High<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV class=Section1>
<P class=MsoNormal><FONT face=Tahoma size=2><SPAN
style="FONT-SIZE: 11pt; FONT-FAMILY: Tahoma">Hi,<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Tahoma size=2><SPAN
style="FONT-SIZE: 11pt; FONT-FAMILY: Tahoma"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Tahoma size=2><SPAN
style="FONT-SIZE: 11pt; FONT-FAMILY: Tahoma">I’ve compiled svn rev 1450 and
tried to install on ws2k3<FONT color=blue><SPAN
style="COLOR: blue">/</SPAN></FONT>ws2k8 x64 with connect<FONT
color=blue><SPAN style="COLOR: blue"> </SPAN></FONT>x<FONT color=blue><SPAN
style="COLOR: blue"> HCA</SPAN></FONT>. The BSOD occur<FONT color=blue><SPAN
style="COLOR: blue">s</SPAN></FONT> during the installation.<FONT
color=blue><SPAN style="COLOR: blue"> The last revision we tested was 1421.
I’ve recompiled few revisions and found that changes between rev 1434 and 1435
cause the bugheck.<o:p></o:p></SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Tahoma color=blue size=2><SPAN
style="FONT-SIZE: 11pt; COLOR: blue; FONT-FAMILY: Tahoma"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Tahoma color=blue size=2><SPAN
style="FONT-SIZE: 11pt; COLOR: blue; FONT-FAMILY: Tahoma">Bottom line -
connect x functionality is broken in revision
1435.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Tahoma size=2><SPAN
style="FONT-SIZE: 11pt; FONT-FAMILY: Tahoma"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Tahoma size=2><SPAN
style="FONT-SIZE: 11pt; FONT-FAMILY: Tahoma">Regards,<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Tahoma size=2><SPAN
style="FONT-SIZE: 11pt; FONT-FAMILY: Tahoma">Anatoly.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Tahoma size=2><SPAN
style="FONT-SIZE: 11pt; FONT-FAMILY: Tahoma"><o:p> </o:p></SPAN></FONT></P></DIV></BLOCKQUOTE></BODY></HTML>