[openib-general] [RFC] send side QP redirection

[Sean Hefty]

>> struct ib_mad_av {
>> 	struct ib_ah *ah;
>> 	u32 remote_qpn;
>> 	u32 remote_qkey;
>> 	u16 pkey_index;
>> };
>
>What about SL and the other redirect GRH fields (TC and FL) ?

These would have been specified through the ib_ah_attr when the destination was
added.  I think that these are the four fields needed to allocate and send a
MAD.  To clarify, an ib_mad_av specifies a <mgmt_class, lid, gid> tuple, which I
refer to as a destination.


>> struct ib_mad_av* ib_insert_mad_dest(struct ib_mad_agent*,
>> 				     struct ib_ah_attr*,
>> 				     mgmt_class, lid, gid, pkey);
>
>So the last three parameters are transferred from the received
>ClassPortInfo.

Not exactly, the destination is inserted based on mgmt_class, lid, and gid
before sending any request.  These are likely coming from a path record.  The
pkey is translated into an index and returned as part of the MAD address vector.
Basically, a user would call this routine in place of ib_create_ah.


>> struct ib_mad_av* ib_get_mad_av(struct ib_mad_agent*,
>> 				mgmt_class, lid, gid);
>
>I'm missing why LID and GID are needed here and why pkey is missing.
>Is there more than 1 struct ib_mad_av allowed per mgmt_class ?

There will be one ib_mad_av per mgmt_class per remote destination.  A remote
destination is identified by the class and LID.  (For the initial
implementation, the GID will essentially be ignored.)


>> /*
>>   * TBD: need to determine when to remove a destination.  Can remove
>>   * always if references go to 0.  Can add a delay before removal.  Can
>>   * maintain destinations that have been redirected. ?
>>   */
>> void ib_free_mad_av(struct ib_mad_av*);
>>
>> ib_redirect_mads(struct ib_mad_av*, struct ib_mad_recv_wc*,
>> 		 struct ib_class_port_info*);
>>
>> /* GID redirection with unknown LID would be deferred */
>> int ib_get_mad_redirect_path(...);
>
>Just curious why ?

The spec permits redirection to another GID without specifying the LID.  This
requires that the requester send a path record query to the SA to obtain the
rest of the information before it can send to the redirected QP.  This results
in an asynchronous operation, which becomes more difficult to deal with.  The
original MAD and others destined for that QP now need to be queued until the
query completes and the full destination is known.

>> Questions or thoughts?
>
>Is multiple redirection handled by this ?

It should be.  ib_redirect_mads uses the existing ib_mad_av to record that
redirection has occurred.  This should allow for multiple redirection.


>Would these semantics need to be extended to user space by user_mad ?

I can't think of a reason why they couldn't.  It would only be called by
userspace clients that initiate sending MADs.

- Sean