[openib-general] multicast code/merge status

[Hal Rosenstock]

On Wed, 2007-01-10 at 13:47, Or Gerlitz wrote:
> On 1/10/07, Sean Hefty <mshefty at ichips.intel.com> wrote:
> >> How about adding sendonly param to the ABI and having the ucma kernel
> >> code returning -EINVAL if someone tries to set it to true. Such code can
> >> be pushed to 2.6.21 and when you have the time to complete the
> >> implementation you can complete this?
> 
> > I don't think adding this is a huge deal; I just haven't gotten to it yet.
> > However, I'd like to make sure there's enough time once the change is made to
> > verify that we have the right result before pushing it upstream.
> 
> Compared to the amount of functionality (ib_sa ref mcast counting and
> rdma cm u/k mcast API) provided by this code, i really think it can be
> done in two phases, first push a code that does not support sendonly
> and later once its implemented push a patch that adds this.
> 
> Anyway, per the best of my knowledge you would not be able to fully
> verify the "right" result , i understand the opensm does not really
> support a sendonly join (*) in the sense of that it does not support
> configuring "one-way" MFTs for a subset of the branches/leaves of the
> multicast group spanning tree, but this is orthogonal to your code.

Yes, this is orthogonal and only affects strict conformance vis a vis
the reception of packets for this group which would be discarded anyhow.
This is an optimization provided by the spec.

> (*) there are some more issues here which need to be addressed, see
> for example the "Some SMs don't support send-only yet" weird comment
> at ipoib_mcast_sendonly_join()

It's more likely an SA issue but I'm only guessing... It may also be
historical...

-- Hal

> > Woody has also seen this issue.  And of course, I can't reproduce it on my
> > systems, but I'm actively looking into the problem.  It looks like some sort of
> > issue with ipoib trying to join a non-existent multicast group.
> 
> mmm, weird, are there active rdmacm multicast consumers involved in
> this settings?
> 
> >> Looking on the code, i understand that if an multicast consumer attempts
> >> to join a group for which another consumer is already joined then it
> >> just gets the group params, that is the mgid is your discriminator (with
> >> the exception of an all zeros mgid which has a different treatment)
> >> which makes much sense to me.
> 
> > Not exactly.  The rdma_cm consumer gets the group parameters for the ipoib
> > broadcast group.  It uses this information as a template for joining new groups.
> 
> OK, got you at last (sorry but i have somehow ignored the call to
> ib_addr_get_mgid() at the rdmacm code). So to achieve interop with
> IPoIB all we need to do is remove the rdmacm signature bit and not to
> over-write the rdmacm qkey on the the qkey of the ipoib ipv4 broadcast
> group, are you ok with that?
> 
> > One issue is that an rdma_cm consumer can first allocate a UD QP to use with UD
> > traffic.  When it later joins a multicast group, the qkey must be the same.  How
> > does ipoib handle this?
> 
> Well, ipoib first sets a zero qkey into its qp in ipoib_init_qp() and
> later in ipoib_mcast_attach() does another qp modify providing
> priv->qkey which is the ipv4 broadcast group one, the rdma_cm can
> mimic this behaviour for qps created with rdma_create_qp and also for
> those created by the user.
> 
> >> Since for our apps needs we do intend to join the 224.0.0.1 group,
> >> resolving a) above is fine for us --> we will join 224.0.0.1 above,
> >> provide the qkey to the rdma cm and it will join to the other group (eg
> >> 224.5.5.5) with this qkey.
> 
> > I'm not completely following you on this yet.
> 
> forget this, there is no need in such tricks since the rdmacm has the
> ipv4 broadcast group qkey.
> 
> >> can you remind me what the idea/trick here, aren't you supposed to
> >> generate an mgid for this case?
> 
> > This either returns an existing MCMemberRecord that this node has joined, or it
> > fills out an MCMemberRecord that can be used to join a new group.  If the mgid
> > is zero, the SA will assign one.
> 
> thanks
> 
> Or.
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>