[openib-general] Re: RMPP

[Hal Rosenstock]

On Wed, 2005-05-04 at 13:43, Sean Hefty wrote:
> Hal Rosenstock wrote:
> > Hi Sean,
> > 
> > In addition to passing the hdr_len and data_len to ib_create_send_mad: 
> >                 rmpp_mad->rmpp_hdr.paylen_newwin = cpu_to_be32(hdr_len -
> >                         offsetof(struct ib_rmpp_mad, data) + data_len);
> > That's not the "real" payload length that would go into the packet; just
> > one segment's worth of class header. Correct ?
> 
> There are two sizes needed by the RMPP code.  First, it needs the total 
> length of the data buffer, which includes any necessary padding.  This is 
> set in the scatter-gather entry for the work request.  Second, it needs to 
> know how much of the data buffer contains valid user data.

Got it.

> ib_create_send_mad returns the length of the of the valid user data in a 
> slightly encoded format.  It subtracts the size of the RMPP and common MAD 
> headers from the size (see below).  The intent is that the payload length 
> value is set to the correct value for MADs that are a single segment in length.

Right. That might be a better way of expressing it than how I did :-)

> -------------
> MAD header
> -------------
> RMPP header
> -------------
> SA/Vendor hdr
> -------------
> User data
> -------------
> Pad
> -------------
> 
> sge length = size from MAD header to end of pad.
> payload = size from SA/Vendor hdr to end of user data.
> 
> > If the above is correct, I'm not sure the actual payload lengths in the
> > DATA packets are correct. I also see the padding on the last segment of
> > a multipacket send not cleared (I integrated the part of your patch
> > relating to the pad calculation).
> 
> I will run some tests and verify the payload values using madeye.  I'm not 
> sure why the padding isn't cleared.  It may be an indication that 
> create_send_mad isn't allocating the correct pad size.

Thanks.

> > In one test case, I see the first RMPP DATA segment sent and there is no
> > response (ACK) from the receiver (this was due to a (receiver) test
> > program issue). The transmitter retry depends on retries in the send_wr
> > ud structure. Does it need to known/Is there a way to know that this
> > failed (no ACK, etc.) when retries are exhausted or is this reliant on
> > the receiver rerequesting or is the entire RMPP transaction treated like
> > a UD send (e.g. unreliable) ?
> 
> The RMPP send is treated as reliable, even if no response is expected.  If 
> the send is not ACKed completely, the request will timeout, and a send 
> failure is reported to the user.  If the send completes successfully, then 
> the user knows that it was received by the remote side; although, there's no 
> guarantee that it was processed by anyone.

Is this on any (RMPP) transmits or just requests ? I will check to see
if I can see this.

> The code will retry a given segment the number of times specified by the 
> user.  If forward progress is made on the send, the retry count is reset.

for each subsequent segment. Makes sense.

> > Here's a summary of changes so far:
> > 1. ib_create_send_mad either needs an additional parameter (RMPP active
> > in current send packet) or the paylen_newwin needs to be set by the user
> > outside of this routine.
> 
> I have this on my short term to-do list.  I'm just not sure on the best 
> approach yet...

OK. I'll wait. I have my workaround right now.

> > 2. Some minor ib_mad.h commentary changes for more clarity on the
> > assumptions of the RMPP API.

Thanks.

-- Hal