[openib-general] question regarding GRH flag in ib_ah_attr

[Hal Rosenstock]

On Fri, 2006-05-12 at 13:10, Jason Gunthorpe wrote:
> On Fri, May 12, 2006 at 08:11:17AM -0400, Hal Rosenstock wrote:
> 
> > > To allow what Roland is talking about you need an unambiguous
> > > mechanism where the SA can signal to the client that the path
> > > needs a GRH.
> > 
> > Ah, you are referring to the SA path record response not the request.
> 
> Yes.. Though I think we are still talking about different things in a
> few places ;>
> 
> How about this, how do you see this scenario:
> 
> 1) Client gets a DGID from 'someplace'
> 2) Client sends a SA query to resolve the DGID to a Path Record
> 3) Client configures a QP based on the Path Record
> 
> Now, the question I'm interested in is this:
>   During step #3 what test should the client apply to determine if a 
>   GRH should be used with the QP.
> 
> Other issues around the GRH like management MAD responses use and
> multicast I feel are well specified and don't need more consideration.

Thanks for clarifying.
 
> > > Think of it the other way, HopLimit < 2 means it _can't_ be forwarded
> > > off subnet, so that result from the SA should _always_ cause the
> > > requesting client to not use a GRH for that path.
> > 
> > Not always true in terms of local subnet (multicast and management MAD
> > response exceptions).
> 
> Yes, but these are well specified. Multicast must always have a GRH.
> MAD requests are covered under my scenario above and MAD responses
> to MAD requests with GRH's are specified to use the GRH and set the
> HopLimit = 0xFF.

Where does the spec say HopLmt needs to be 0xFF for multicast ?

> Also, I would assume when building a router that multicast packets
> with a hop limit of 0 are non-forwardable based on the rules in IBA.

0 or 1 hop limit for both unicast and multicast

> > Are you saying HopLimit is supplied to the SA in the request ? It could
> > be but it's optional in general. In the router case, an off subnet DGID
> > should be sufficient. I would think the HopLimit (as well as the other
> > GRH fields) would need to be returned by the SA to the client.
> 
> Talking about a request for a Path to the SA from a client now:
> I would suggest that if the client wishes to restrict itself to paths
> that are only on-link then it could send a SA request with the
> path record HopLimit=0.

Yes (or HopLimit=1).

>  A SA request with HopLimit=* (masked out
> of component mask) should let the SA return routed paths.

Yes.

> I also think that the SA response should have a HopLimit of 0 for
> local paths

1 would also be valid here too.

>  and a HopLimit >= 2 for routed paths.

Yes.

> However, I can't find any wording in IBA that would require this
> behavior.

In terms of the SA responses to Path/MultiPathRecord requests, the
HopLimit is required to be filled in in the response. Is that what you
are asking ? It's up to the SA to determine this and for the client to
use the values returned subsequently just as it does for DLIDs, SLs,
etc.

> > Not sure exactly what you mean by full control over the routing header
> > (GRH). The SA supplies the info for the headers to the client and the
> > client is responsible for putting the correct info in the headers. Do
> > you mean supplies sufficient info for the client to do this correctly ?
> > If so, I agree.
> 
> As far as I can see IBA includes all header information for the GRH
> and LRH in the PathRecord response. It does not define a how to
> determine if the path described by a PathRecord response requires
> a GRH or not.

I think the rules are there:
Multicasts always have GRH.
Unicasts off subnet have GRH and on subnet they are optional.

Off subnet is either determined by the prefix comparison or HopLimit >=2
in the response from the SA. The latter is implied by C8-16 on p. 229.

-- Hal

> Thanks,
> Jason