[ofw] Re: Completion with bad status: IBV_WC_RETRY_EXC_ERROR

[Diego Guella]

[resend 1 because of attachment problems]


Hi Dotan,
 
About the failure in RDMA Read, do you mean the garbage data that's printed out?
I don't mind about that for now, that's expected because the buffer isn't initialized in the client.
 
 
I already thought about endianness of the LID, and tried to change endianness but with no success.
 
But now I modified the code in both Windows and Linux to send out the LID in network order.
The Windows side does nothing(sends and receives the LID as is).
The Linunx side changes endianness of the LID before sending, and changes endianness of the received LID.
(Attached new sources and executables)
 
 
But the problem is still there.
 
 
Do you have any other ideas?
 
 
Thanks,
Diego

  ----- Original Message ----- 
  From: Dotan Barak 
  To: Diego Guella ; Tzachi Dar ; Fab Tillier 
  Cc: ofw at lists.openfabrics.org 
  Sent: Wednesday, November 14, 2007 9:57 AM
  Subject: RE: Completion with bad status: IBV_WC_RETRY_EXC_ERROR


  Hi.

  I checked your code in Linux and you have a failure in RDMA Read...

   But for your problem: in windows the LID of the port is in network order and in Linux the LID of the port is in host order.
  (order means endianess).


  Fixing this should solve the problem ...


  Dotan




----------------------------------------------------------------------------
    From: Diego Guella [mailto:diego.guella at sircomtech.com] 
    Sent: Wednesday, November 14, 2007 10:29 AM
    To: Dotan Barak; Tzachi Dar; Fab Tillier
    Cc: ofw at lists.openfabrics.org
    Subject: Re: Completion with bad status: IBV_WC_RETRY_EXC_ERROR


    Hi Dotan,

    I apologize for that silly mistake, obviously WR_SEND is different from IBV_WR_SEND, and the same was for WR_RDMA_READ, etc. etc...

    So, I removed <iba/ib_types.h> from the includes, to make sure I don't use them.

    Now the Linux program works with send/recv and rdma to himself (daemon/client on the same machine), but I still get the same error when I try communication between Windows/Linux.
    The error applies to SEND, RDMA_WRITE, RDMA_READ, and using the daemon both on Linux or Windows.

    Attached are the new sources (note that Windows sources aren't changed).



    Thanks,
    Diego



      ----- Original Message ----- 
      From: Dotan Barak 
      To: Diego Guella ; Tzachi Dar ; Fab Tillier 
      Cc: ofw at lists.openfabrics.org 
      Sent: Tuesday, November 13, 2007 3:13 PM
      Subject: RE: Completion with bad status: IBV_WC_RETRY_EXC_ERROR


      o.k., I passed that and the code compilation stage ....


      Just a small note:
      In the modify_qp: qp_access_flags is only for supported remote operations, so IBV_ACCESS_LOCAL_WRITE should be removed


      I understand what the root cause of the problem is: you took a code from windows and moved only PART of the code to Linux.

      For example: WR_SEND is defined in iba/ib_types and has the value 1 (which is RDMA WRITE_WITH_IMM in Linux)
      so actually, you did RDMA write with rkey and remote address with undefined values.
      (this is only an example for the corruption that happens during the test execution because of this issue)

      The code passed compilation because you included iba/ib_types.h (in types.h) in Linux too.
      This file should not be included in Linux (unless you really need it ...)

      All of the structures, functions, enumerations in Linux verbs start with IBV_ or ibv_.


      This should fix the test problems (I hope ...)


      thanks
      Dotan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofw/attachments/20071114/c2da9447/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: winib_1.3_tests.zip_
Type: application/octet-stream
Size: 121480 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/ofw/attachments/20071114/c2da9447/attachment.obj>