[ofa-general] Sending two integers via RDMA_WRITE

[Dotan Barak]

Zafar Gilani wrote:
> Thanks for the reply. I read your name under the Author for may be all 
> the IBV structs/operations at linux.die.net <http://linux.die.net>. So 
> I am highly impressed by the work (can only dream of it myself). :)
thanks ...
>
> 1. I don't know why the original author (Roland Dreir) has used 
> ntohl() for rkey and remote_addr. Though to use it on buffer (buf) is 
> essential in order to transfer the byte order from network to host.
I need to check the test code to check the reason....
>
> 2. I am using QP type RC for reliable connection.
This means that if there is an error you will get a bad completion
(it you would have use UC QP, in case of an error in the receiver side, 
the packet would have dropped).
>
> 3. Yes I am checking for that but the code gets stuck before that, 
> around when I call ibv_get_cq_event to wait for next completion event 
> in the event channel. I think the second write (second iteration of 
> for loop) is not working properly since when I try to send buf[0] via 
> RDMA_WRITE and buf[1] via SEND then it works fine.
Using ibv_get_cq_event can be tricky, you must arm the CQ (call 
ibv_req_notify_cq) BEFORE a completion can enter to that CQ.
To make things more clear, why won't you just poll the CQ for completion 
directly? (without using the CQ events)

I believe that you will get a completion with error...

Dotan
>
> The code I am walking about:
>
>  while(1) {
>     if(ibv_get_cq_event(comp_chan, &evt_cq, &cq_context)) // here it 
> gets stuck
>         return 1;
>
> // does not print this
> printf("after get_cq_event\n"); fflush(stdout);
>
>     if(ibv_req_notify_cq(cq, 0))
>         return 1;
>
> printf("after req_notify_cq\n"); fflush(stdout);
>
>     if(ibv_poll_cq(cq, 1, &wc) != 1)
>         return 1;
>
> printf("after poll_cq\n"); fflush(stdout);
>
>     if(wc.status != IBV_WC_SUCCESS)
>         return 1;
>
> printf("after wc.status\n"); fflush(stdout);
>
>     if(wc.wr_id == 0) {
>         printf("%d\n", ntohl(buf[0])); fflush(stdout);
>         return 0;
>     }
>   }
>
> Your thoughts on this?
>
> Thanks,
> zafar