[openib-general] [PATCH 1/2] perftest: enhancement to rdma_bw to allow use of RDMA CM

[Pradipta Kumar Banerjee]

Michael S. Tsirkin wrote:
> Quoting r. Pradipta Kumar Banerjee <bpradip at in.ibm.com>:
>> Subject: Re: [openib-general] [PATCH 1/2] perftest: enhancement to rdma_bw to allow use of RDMA CM
>>
>> Michael S. Tsirkin wrote:
>>> Thanks for following up on this, on the surface looks good.
>>> I'm busy at the moment, but a quick question:
>>>
>>> Quoting r. Pradipta Kumar Banerjee <bpradip at in.ibm.com>:
>>>> +static void pp_wait_for_done(struct pingpong_context *ctx)
>>>> +{
>>>> +	struct ibv_wc wc;
>>>> +	int ne;
>>>> +
>>>> +	do {
>>>> +		usleep(500);
>>>> +		ne = ibv_poll_cq(ctx->rcq, 1, &wc);
>>>> +	} while (ne == 0);
>>> I wander whether we can get rid of the rcq and an extra send at the
>>> end by simply closing the QP, and ignoring the possible completion with error
>>> for the last round. I think this would be a nice simplification.
>>> What do you think?
>>>
>> Indeed we can get rid of the extra send at the end (pp_wait_for_done and 
>> pp_send_done can be removed). As for the rcq need to think about it. Its being 
>> used to synchronize the client and the server so that the client sends the first 
>> message.
> 
> Why do we need an RC send for this? CM or socket message should be enough, no?
> 
Here is a case where socket message doesn't suffice -
Ammasso iWARP cards create two interfaces - one the usual ethernet interface 
(ethX) and another iWARP interface (iwX).
The RDMA protocol is run over the iWARP interface which doesn't allow for normal 
socket based communication. So if we want to use sockets for exchanging the 
setup info we need to add one more option to the rdma_bw.c, which will allow to 
use the IP addr bound to the ethX interface for socket based communication and 
the IP addr bound to the iWARP interface for RDMA operations.
IMO using rcq seems to be a generic and better solution.
What do you say ?

Thanks,
Pradipta Kumar.