[openib-general] Re: [PATCH] [RFC] - example user moderdmaping/pongprogram using CMA

[Michael Krause]

At 11:04 AM 2/8/2006, Michael S. Tsirkin wrote:
>Quoting r. Steve Wise <swise at opengridcomputing.com>:
> > Subject: Re: [openib-general] Re: [PATCH] [RFC] - example user 
> moderdmaping/pongprogram using CMA
> >
> > On Wed, 2006-02-08 at 19:10 +0200, Michael S. Tsirkin wrote:
> > > Quoting r. Sean Hefty <sean.hefty at intel.com>:
> > > > Subject: RE: [openib-general] Re: [PATCH] [RFC] - example user mode 
> rdmaping/pongprogram using CMA
> > > >
> > > > >Steve, looks like you have at most a single receive work request 
> posted at the
> > > > >receive workqueue at all times.
> > > > >If true, this is *really* not a good idea, performance-wise, even 
> if you
> > > > >actually have at most 1 packet in flight.
> > > >
> > > > Can you provide some more details on this?
> > >
> > > See 9.7.7.2 end-to-end (message level) flow control
> > >
> >
> > I just read this section in the 1.2 version of the spec, and I still
> > don't understand what the issue really is?  9.7.7.2 talks about IBA
> > doing flow control based on the RECV WQEs posted. rping always ensures
> > that there is a RECV posted before the peer can send.  This is ensured
> > by the rping protocol itself (see the comment at the front of rping.c
> > describing the ping loop).
> >
> > I'm only ever sending one outstanding message via SEND/RECV.  I would
> > rather post exactly what is needed, than post some number of RECVs "just
> > to be safe".  Sorry if I'm being dense.  What am I missing here?
> >
> > Steve.
> >
>
>As far as I know, the credits are only updated by the ACK messages.
>If there is a single work request outstanding on the RQ,
>the ACK of the SEND message will have the credit field value 0
>(since exactly one receive WR was outstanding, and that is now consumed).
>
>As a result the remote side withh "think" that there are no
>receive WQEs and will slow down (what spec refers to as limited WQE).

Correct.   The ACK / NAK protocol used by IB is used to return credits.  In 
order to pipeline to improve performance, then you must post multiple 
receive work requests in order to account for the expected round trip time 
of the fabric and the associated CA processing.

Mike 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060208/24e05330/attachment.html>