[openib-general] [PATCH 1/2] perftest: enhancement to rdma_bw to allow use ofRDMA CM

[Steve Wise]

On Thu, 2006-07-06 at 17:57 +0300, Michael S. Tsirkin wrote:
> Quoting r. Steve Wise <swise at opengridcomputing.com>:
> > Subject: Re: [PATCH 1/2] perftest: enhancement to rdma_bw to allow use ofRDMA CM
> > 
> > On Thu, 2006-07-06 at 09:20 +0300, Michael S. Tsirkin wrote:
> > > Quoting r. Steve Wise <swise at opengridcomputing.com>:
> > > > For instance, in the bi-directional bandwidth tests, one side could
> > > > finish the test and issue an rdma_disconnect() while the other end still
> > > > has pending rdma writes.  This is an error condition in iWARP.  So they
> > > > sync up at the end with a message exchange before disconnecting.
> > > > 
> > > > Make sense?
> > > 
> > > I see. Is this the only message exchange there is?
> > 
> > 
> > There are 2 message exchanges to comply with IWARP spec: 
> > 
> > 1) an exchange right after connection setup to force the client to be
> > the first side to send an RDMA message.  So the server waits for a RECV
> > completion and the client posts a "lets start" SEND.
> 
> I don't get this one.  Why can't we just look at command line arguments for
> this?  Whoever got the remote IP is the client.  What do you mean by "force
> client to be the first"?

This is an iWARP spec/compliance issue.  The client MUST send the first
RDMA message.  The server CANNOT send an RDMA message until one is
received from the client.  We've discussed this before on the openib
list.  (If you want gory details on this, I'll find the original thread
where we discussed this). 

I'm not arguing that this is a reasonable restriction, but it is what it
is.  And if the perftests are going to run over rnics, they should have
this code...


Steve.