[openib-general] Re: ib_mad.h ib_mad_agent.hi_tid

[Hal Rosenstock]

On Wed, 2004-09-01 at 13:02, Sean Hefty wrote:
> On Wed, 01 Sep 2004 13:34:11 -0400
> Hal Rosenstock <halr at voltaire.com> wrote:
> 
> > The one additional complication appears to be when a REJ is sent by the
> > active side from the Timeout or REP Wait state as a result of CM
> > protocol timeout. The CM protocol timeout is indicated by Message
> > REJected = 2. Reason code is 4 for timeout. Does the access layer need
> > to make exceptions for those REJs ?
> 
> ...gurgle...

Was that the proverbial straw :-( 

> Here's what we did for the SF stack.  All CM MADs were routed at the 
> receive side using a dispatch table based on "unsolicited" registration.  
> (There was a special check for the CM class.)
> 
> The impact was that a client other than the CM could not initiate a 
> connection request by sending a REQ directly.  It would be nice to avoid 
> this restriction, which should allow multiple "active" side CMs to co-exist, 
> but definitely isn't a requirement, and probably involves embedding a good 
> deal of the CM protocol into the access layer.  

As long as the CM clients use the hi_tid assigned by the access layer in
forming their TIDs, I don't see a problem with multiple active CMs.

> On the send side, 
> the access layer did not change the TID for CM MADs.
> 
> My preference is for the access layer to know as little about the CM protocol 
> as possible.  

Mine as well.

> I think it would be nice if the access layer could examine 
> something as simple as the R bit to determine if it needed to set the upper TID, 
> but that's not possible given the CM protocol.  

This may not only be an issue with CM but also vendor and application
classes too (as they too can use Send methods (non request/response)).
The other classes use of Send is more abstract than CM.

> As an alternative, we could extend the mad_flags to indicate if sent a MAD 
> should be treated as: default (no flag set, so check the R bit), solicited 
> (do not set TID), or unsolicited. (set TID).
> 
> Of course, if the client needs to set a flag, it could just set the TID directly 
> and be done...  Maybe this is the way to go on the send side.
> 
> What are your thoughts on this?

I'm back to where I was on this as adding a flag seems of minimal
benefit as you point out. So I would have the access layer set
unsolicited TIDs with the exception of Send methods (CM and other
clients using the Send method need to handle their own TIDs on send).
If that is deemed too unclean, then all send TID handling is back at
the client :-(

Also, should the access layer validate that the client uses the hi_tid
supplied in the mad_agent in the TIDs of sent MADs ?

-- Hal