[openib-general] Some ib_mad.h Redirection Comments

[Yaron Haviv]

On Friday, August 06, 2004 9:10 PM, Sean Hefty wrote:
> On Thu, 05 Aug 2004 20:13:26 -0400
> Hal Rosenstock <halr at voltaire.com> wrote:
> 
>> I don't think this by itself forces redirection above the API. I
>> think it can still be handled transparently inside the GSI.
> 
> Trying to handle this "transparently" in the GSI puts too much policy
> in the GSI.  I think that it's necessary for the clients to manage
> their QPs, not the access layer.  

I think Hal is focused on the Request side any Sean on the Response
(Server) side 
I'm not sure there is argument that the Server side owns the QP's (other
than QP1 owned by the GSI)

>>> The client would be responsible for caching the redirection.
>> 
>> This seems like a commonality which can be pushed down into the
>> access layer and makes more sense when the access layer is handling
>> the retransmissions on redirect. If it isn't, it makes less sense.
> 
> Sending to a redirected QP involves setting the remote_qpn,
> remote_qkey, and pkey_index in the ib_mad_send_wr to different
> values.  It's trivial for a client to set this.  

I don't see what is the value in having every potential GSI client
implement a common functionality of identifying it's a Redirect, and
resending the request to the new location, why not just have the GSI
layer do that common functionality for all the consumers ?
I think it is also trivial for the GSI to implement such functionality 

> 
>> So back to that issue: if the access layer is already matching
>> responses to requests (when timeout is enabled), I don't see why it
>> doesn't handle the retransmission on receipt of redirect so each
>> client doesn't need to implement this.
> 
> Currently, the access layer is not retransmitting MADs outside of
> RMPP.  When a response comes in, both the request and response may be
> handed to the user.  If the response is a redirect, the client can
> cache the redirection information and re-issue the request.  All
> future requests sent by that client can now automatically go to the
> correct location without the GSI having to snoop the destination,
> lookup whether that destination has been redirected, and modifying
> the outbound MAD.       

This doesn't handle cases were there are multiple clients that access
the same remote service (e.g. multiple SA clients), if the GSI layer
implement the caching, all clients can benefit from it, not to mention
having a central implementation.
If your main concern is CM (that has a single client per node, multiple
remote servers, and some different behavior) we can treat it as an
exception, I'm more focused at things such as distributed SA in such
cases we can cache/lookup based on a class first 

> 
> Are you concerned about duplicating the management of a redirection
> table?  I think that redirection tables must be maintained per
> client.  Would adding support for redirection table management remove
> your concerns?

I thing we are starting to complicate something that was supposed to be
trivial 

If the Requestor side redirect handling is done at the GSI level it
makes the Requestor code trivial:
Just build the proper MAD, send it, and parse the result (or retry)
after the callback  
And not have any potential consumer deal with the same cases and
complicate its code.