[openib-general] user_mad check question

[Rimmer, Todd]

> From: Sean 
> 
> 
> Can sends ever be used with RMPP?

Yes, however no present classes define this, but that does not rule out
future classes.  13.6.2 indicates the method field may be used by an
RMPP class in any way.  

> 
> > One such class which can use SEND to hold requests and their
responses
> > is the BMA.  So can the SNMP tunnel and the CM (although CM is not
> > usually done from user space).  For the BMA case, a request can be:
> > Class=BM
> > Method=SEND
> > AttributeModifier has BM RESPONSE bit set
> 
> Ugh.  This makes it really hard to determine if a send is a new
operation.

The BM case can be tested for specifically, however future classes or
vendor specific classes are tougher.

The approach we took in our proprietary stack was to classify messages
as request/responses based on presently defined classes (BM was only
special case needed).  We permitted any consumer to issue requests (and
created unique tids which helped the stack direct the response back to
the client).  For a consumer to get inbound requests or any other
unsoliticed messages (such as SEND), it had to register with the stack
for that class and we only permitted one such consumer to register per
class.  This permitted those messages to be directed solely based on
class.

In this context consumer could be kernel or user level.  For example the
CM and PMA were each kernel level consumers.  A user level consumer was
also possible.

In this context we defined a request as:
(R bit = 0 && ! TRAP_REPRESS && ! SEND) || (Class=BM && SEND &&
AttributeModifier BM Response bit not set)

We defined a response as:
((R bit set || TRAP_REPRESS) && ! SEND) || (Class=BM && SEND &&
AttributeModifier BM Response bit set)

While similar these are not strictly complements of eachother.  There
are some messages (such as SEND for the CM) which will be classified as
neither a request nor a response.  For such messages the TID cannot be
safely altered and those messages would be routed only by Class to
exactly 1 consumer so registered.

This approach allowed multiple clients for a given standard class (such
as the BM, PM, etc) to exist on a given node.  However only a single
agent could run on a node.  Vendor specific classes choosing to use send
would be limited to a single client OR agent per node.  To date we have
not found any cases where this limitation was an issue.

Since the above algorithm for request/respone identification is a little
tricky and subject to possible extension in the future (as new classes
are defined), I would recommend putting this in a inline function or
macro alongside the datatypes for mad packets.

Todd Rimmer