[ofw] Re: Completion with bad status: IBV_WC_EXC_RETRY_EXC_ERROR

[Diego Guella]

Hi Stan,
Thanks for your reply.
Please see my replies inline.


----- Original Message ----- 
>From: "Smith, Stan" <stan.smith at intel.com>
>
>You never will until the Windows IB CM (Communications/Connection
>Manager) is compatible with the Linux IB/RDMA CM...

I'm not using the standard CM. When I talk about "CM" I use the double 
quotes because I'm using a TCP connection (over IPoIB) to exchange the CM 
data.

>Currently the only working approach is to exchange QP information via a
>TCP/IP socket connection and then transition the QPs manually; rather
>advanced IB programming.

This is actually how I am doing it :)
It can be "advanced", but when I started to work on Infiniband this was the 
only way to do this, and I have already a test application on Linux that 
works in this way.

>Stick with IPoIB/SDP, far easier thus allowing you to concentrate on the
>_real_ problem you are trying to solve.

Our real problem, is that we need at least one of the machines to be on 
Windows, because of some software that is available only in this platform.
And so we need to communicate to Windows at IB speeds.

On Linux, with Verbs and RDMA, I've been able to exchange data at ~1340MB/s 
on a DDR card, and 0 CPU used. Currently IPoIB can't even think of approach 
this speed, and SDP, even if it is able, will use 100% CPU.
Correct me if I'm wrong, but this seems to me how thing are today.

I tried the IPoIB way, but it is too slow.
SDP don't seem to work.
The verbs way works on Windows<->Windows, works on Linux<->Linux, but I 
can't get it working Windows<->Linux because of this RETRY_EXC_ERROR, bu I 
can't figure out why :(



Thanks,
Diego