[openib-general] [PATCH] added comments to ib_mad.h - minor update

[Sean Hefty]

On Thu, 05 Aug 2004 20:44:33 -0400
Hal Rosenstock <halr at voltaire.com> wrote:
> This answered a number of my previous questions. Code talks...

I want to make sure that we can get agreement on the API.  I do think that we're getting close.
 
> ib_mad_agent has lo_tid of 32 bits. What are the other 32 bits of tid ?
> I vaguely recall some email (thread) went by on this.

The TID was going to be split between the client and the access layer.  The access layer would use the ib_mad_agent to set its portion of the TID on outgoing requests.  This allows it to route received responses to the proper client, but ensures that MADs sent from multiple clients do not conflict on TIDs.
 
> ib_mad_send_wr.sg_list indicates first entry must reference a data
> buffer of 256 bytes. Is the "base" RMPP header in it ? Which fields
> must be filled in by the client (RMPPActive, RRespTime, and length) ?
> Will length be in the first segment only (total length) or also in the
> last segment ?

Because the spec positioned the RMPP header in the middle of user-data (i.e. after the standard MAD header), I think that the most efficient way to handle sending a MAD is for the client to hand the access layer a buffer that contains both the MAD header and RMPP header.  For most MADs, this results in a send work request that uses a single sg-entry.

If we agree on this, then the intent here is that we don't want the user handing the access layer the RMPP header split across multiple data buffers.  The real restriction is that the first sg-entry should reference both the MAD and RMPP headers.  I think that the user only needs to set RRespTime and RMPPActive flag in the RMPP header if using RMPP.  If RMPP is not used, they should set all fields to 0.

> On the receive side, we need to handle either if we have
> to deal with non OpenIB implementations :-(

On the receive side, we control the data buffers, so this isn't an issue.  We should just post receive buffers of 256 + sizeof(grh).

> What about subsequent entries in the s-g list for send ? Are they also
> constrained to be 256 bytes or something else ? I would presume
> RMPP would rewrite the RMPP header based on the first header and
> update the appropriate fields.

We can be as flexible or restrictive as we want to be, I think.  My request (based on feedback) is that we try to minimize the need to perform any data copies.

> Is timeout_ms used for Ttime when it is a RMPP send ?

timeout_ms applies to sends, whereas Ttime applies to the receiver.  We could use the default of 40 seconds as mentioned in the spec, but this seems high to me.  For a received response, timeout_ms should work fine.

> I am still wondering about the RMPP direction switch (IsDS) and whether
> this needs to be exposed somehow.

I don't think that it does.  I don't think anything like this was needed in the sourceforge stack, and the proposed GSI implementation uses the sourceforge RMPP code.  I think that we just need to know if a send requires segmentation, or if a receive requires reassembly.