[openib-general] [RFC] send side QP redirection

[Sean Hefty]

Hal Rosenstock wrote:
>>>>struct ib_mad_av {
>>>>	struct ib_ah *ah;
>>>>	u32 remote_qpn;
>>>>	u32 remote_qkey;
>>>>	u16 pkey_index;
>>>>};
>>>
>>>What about SL and the other redirect GRH fields (TC and FL) ?
>>
>>These would have been specified through the ib_ah_attr when the destination was
>>added.  I think that these are the four fields needed to allocate and send a
>>MAD.  To clarify, an ib_mad_av specifies a <mgmt_class, lid, gid> tuple, which I
>>refer to as a destination.
> 
> Aren't LID and GID also part of ib_ah_attr struct too ?

Yes.  The idea is that <mgmt_class, lid, gid> are used to identify a unique 
remote agent.  See next comment.


>>>>struct ib_mad_av* ib_insert_mad_dest(struct ib_mad_agent*,
>>>>				     struct ib_ah_attr*,
>>>>				     mgmt_class, lid, gid, pkey);
>>>
>>>So the last three parameters are transferred from the received
>>>ClassPortInfo.
>>
>>Not exactly, the destination is inserted based on mgmt_class, lid, and gid
>>before sending any request.  These are likely coming from a path record.  The
>>pkey is translated into an index and returned as part of the MAD address vector.
>>Basically, a user would call this routine in place of ib_create_ah.
> 
> But this call takes an ib_ah_attr struct so I don't see how this call is
> in place of ib_create_ah.

In place of calling ib_create_ah(struct ib_pd*, struct ib_ah_attr*) to obtain 
struct ib_ah*, this call is made.  Since struct ib_mad_av contains struct 
ib_ah*, it can be used in place of it.  As you pointed out above, the lid and 
gid are part of struct ib_ah_attr and can be removed as parameters.


>>>>/*
>>>>  * TBD: need to determine when to remove a destination.  Can remove
>>>>  * always if references go to 0.  Can add a delay before removal.  Can
>>>>  * maintain destinations that have been redirected. ?
>>>>  */
>>>>void ib_free_mad_av(struct ib_mad_av*);
>>>>
>>>>ib_redirect_mads(struct ib_mad_av*, struct ib_mad_recv_wc*,
>>>>		 struct ib_class_port_info*);
> 
> Does this return void or int ?

This will return int or struct ib_mad_av*, since it may fail.  Internally, it 
will allocate a new ib_mad_av, since the existing one may still be in use.


>>>>/* GID redirection with unknown LID would be deferred */
>>>>int ib_get_mad_redirect_path(...);
> 
> I can see why this functionality should be deferred from an complexity
> standpoint. Any idea where this might be used rather than LID
> redirection ?

My guess is only once global routing is defined.

- Sean