[openib-general] [PATCH 1/2] perftest: enhancement to rdma_bw to allow use of RDMA CM

[Steve Wise]

On Wed, 2006-07-05 at 14:29 +0300, Michael S. Tsirkin wrote:
> Thanks for following up on this, on the surface looks good.
> I'm busy at the moment, but a quick question:
> 
> Quoting r. Pradipta Kumar Banerjee <bpradip at in.ibm.com>:
> > +static void pp_wait_for_done(struct pingpong_context *ctx)
> > +{
> > +	struct ibv_wc wc;
> > +	int ne;
> > +
> > +	do {
> > +		usleep(500);
> > +		ne = ibv_poll_cq(ctx->rcq, 1, &wc);
> > +	} while (ne == 0);
> 
> I wander whether we can get rid of the rcq and an extra send at the
> end by simply closing the QP, and ignoring the possible completion with error
> for the last round. I think this would be a nice simplification.
> What do you think?
> 

For IWARP, the ULP must quiesce the QP before moving it to CLOSING (ie
before doing an rdma_disconnect()).  By quiesce, I mean make sure there
are no outstanding SQ WRs and no outstanding RDMA reads in progress.
That's why there is this message exchange at the end of the test.  

For instance, in the bi-directional bandwidth tests, one side could
finish the test and issue an rdma_disconnect() while the other end still
has pending rdma writes.  This is an error condition in iWARP.  So they
sync up at the end with a message exchange before disconnecting.

Make sense?