[openib-general] Re: Re: [PATCH] [RFC] - example user mode rdmaping/pongprogram using CMA

[Steve Wise]

On Wed, 2006-02-08 at 09:51 -0800, Caitlin Bestler wrote:
> openib-general-bounces at openib.org wrote:
> >>> By the way, in case its not clear:  The SEND/RECV exchanges are done
> >>> just to advertise source and sink memory regions, and to indicate
> >>> completion of rdma read and write operations to the peer.  The
> >>> "ping/pong" data is transferred with rdma read and write operations.
> >>> 
> >>> Thanks for the feedback!
> >>> 
> >> 
> >> Code tends to get copied around ... its easy to imagine someone
> >> copying this and measuring the send latency. Just posting many WRs in
> >> the initialization sequence, with no other code changes, will fix
> >> this problem. 
> >> 
> > 
> > Each "ping/pong" iteration with rping is composed of 2 sends
> > on the client side, 2 sends on the server side, plus 1 rdma
> > read and 1 rdma write on the server side.
> > 
> > Again, latency performance (or any performance) isn't a goal
> > of this program.  Testing CMA, CQ and CMA event
> > notifications, and send/recv/rr/rw are the goals.
> > 
> > 
> > snipit from the patch:
> > 
> > +/*
> > + * rping "ping/pong" loop:
> > + *     client sends source rkey/addr/len
> > + *     server receives source rkey/add/len
> > + *     server rdma reads "ping" data from source
> > + *     server sends "go ahead" on rdma read completion
> > + *     client sends sink rkey/addr/len
> > + *     server receives sink rkey/addr/len
> > + *     server rdma writes "pong" data to sink
> > + *     server sends "go ahead" on rdma write completion + *    
> > <repeat loop> + */
> > 
> 
> Why does the server send "go ahead" after rdma write completion?

No particular reason.

> It should be able to just post the send after posting the rdma
> write without waiting. When the rdma write completes has no
> device/transport independent meaning.

You're correct.  It does not need to wait for the rdma write
completion...