<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<STYLE></STYLE>
<META content="MSHTML 6.00.6000.16544" name=GENERATOR></HEAD>
<BODY bgColor=#ffffff>
<DIV dir=ltr align=left><SPAN class=078235508-14112007><FONT face=Arial
color=#0000ff size=2>Hi.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=078235508-14112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=078235508-14112007><FONT face=Arial
color=#0000ff size=2>I checked your code in Linux and you have a failure in RDMA
Read...</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=078235508-14112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=078235508-14112007><FONT face=Arial
color=#0000ff size=2> But for your problem: in windows the LID of the port
is in network order and in Linux the LID of the port is in host
order.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=078235508-14112007><FONT face=Arial
color=#0000ff size=2>(order means endianess).</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=078235508-14112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=078235508-14112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=078235508-14112007><FONT face=Arial
color=#0000ff size=2>Fixing this should solve the problem
...</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=078235508-14112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT> </DIV>
<DIV dir=ltr align=left><FONT face=Arial size=2>Dotan</FONT></DIV>
<DIV> </DIV><BR>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Diego Guella
[mailto:diego.guella@sircomtech.com] <BR><B>Sent:</B> Wednesday, November 14,
2007 10:29 AM<BR><B>To:</B> Dotan Barak; Tzachi Dar; Fab Tillier<BR><B>Cc:</B>
ofw@lists.openfabrics.org<BR><B>Subject:</B> Re: Completion with bad status:
IBV_WC_RETRY_EXC_ERROR<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV><FONT face=Arial size=2>Hi Dotan,</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>I apologize for that silly mistake, obviously
WR_SEND is different from IBV_WR_SEND, and the same was for WR_RDMA_READ, etc.
etc...</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>So, I removed <iba/ib_types.h> from the
includes, to make sure I don't use them.</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>Now the Linux program works with send/recv
and rdma to himself (daemon/client on the same machine), but I still get the
same error when I try communication between Windows/Linux.</FONT></DIV>
<DIV><FONT face=Arial size=2>The error applies to SEND, RDMA_WRITE, RDMA_READ,
and using the daemon both on Linux or Windows.</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>Attached are the new sources (note that Windows
sources aren't changed).</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>Thanks,</FONT></DIV>
<DIV><FONT face=Arial size=2>Diego</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<BLOCKQUOTE dir=ltr
style="PADDING-RIGHT: 0px; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #000000 2px solid; MARGIN-RIGHT: 0px">
<DIV style="FONT: 10pt arial">----- Original Message ----- </DIV>
<DIV
style="BACKGROUND: #e4e4e4; FONT: 10pt arial; font-color: black"><B>From:</B>
<A title=dotanb@mellanox.co.il href="mailto:dotanb@mellanox.co.il">Dotan
Barak</A> </DIV>
<DIV style="FONT: 10pt arial"><B>To:</B> <A
title=diego.guella@sircomtech.com
href="mailto:diego.guella@sircomtech.com">Diego Guella</A> ; <A
title=tzachid@mellanox.co.il href="mailto:tzachid@mellanox.co.il">Tzachi
Dar</A> ; <A title=ftillier@windows.microsoft.com
href="mailto:ftillier@windows.microsoft.com">Fab Tillier</A> </DIV>
<DIV style="FONT: 10pt arial"><B>Cc:</B> <A title=ofw@lists.openfabrics.org
href="mailto:ofw@lists.openfabrics.org">ofw@lists.openfabrics.org</A> </DIV>
<DIV style="FONT: 10pt arial"><B>Sent:</B> Tuesday, November 13, 2007 3:13
PM</DIV>
<DIV style="FONT: 10pt arial"><B>Subject:</B> RE: Completion with bad
status: IBV_WC_RETRY_EXC_ERROR</DIV>
<DIV><BR></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>o.k., I passed that and the code compilation stage
....</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN><SPAN class=625085213-13112007><FONT
face=Arial color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>Just a small note:</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>In the modify_qp: qp_access_flags is only for supported
remote operations, so IBV_ACCESS_LOCAL_WRITE should be
removed</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>I understand what the root cause of the problem is: you
took a code from windows and moved only PART of the code to
Linux.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>For example: WR_SEND is defined in iba/ib_types and has
the value 1 (which is RDMA WRITE_WITH_IMM in Linux)</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>so actually, you did RDMA write with rkey and remote
address with undefined values.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>(this is only an example for the corruption that
happens during the test execution because of this issue)</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>The code passed compilation because you included
iba/ib_types.h (in types.h) in Linux too.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>This file should not be included in Linux (unless you
really need it ...)</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>All of the structures, functions, enumerations in Linux
verbs start with IBV_ or ibv_.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=625085213-13112007><FONT face=Arial
color=#0000ff size=2>This should fix the test problems (I hope
...)</FONT></SPAN></DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT> </DIV>
<DIV><SPAN class=625085213-13112007><FONT face=Arial color=#0000ff
size=2>thanks</FONT></SPAN></DIV>
<DIV dir=ltr align=left><FONT face=Arial size=2>Dotan</FONT></DIV>
<DIV><FONT face=Arial
size=2></FONT> </DIV></BLOCKQUOTE></BLOCKQUOTE></BODY></HTML>