[openib-general] [PATCH 1/2] rdma_cm: add support to join IPOIB multicast groups

[Or Gerlitz]

On 1/24/07, Sean Hefty <mshefty at ichips.intel.com> wrote:
> > However, it is possible that an RDMA_PS_IPOIB consumer would want to
> > talk over ---one-- UD QP with two peers:
> >
> > 1) IPoIB - multicast traffic
> > 2) --another-- RDMA CM consumer - unicast traffic

> My thinking on this was that path record lookup and SIDR resolution isn't part
> of the ipoib protocol, and I wanted to limit the scope of the patch.

indeed they are not part of the ipoib protocol, but the reason there
interop is not possible between PS_IPOIB ID/QP to peer node IPoIB UD -
is much more simple - as of IPoIB address resolution...

The peer IPoIB would send an ARP and then would assume it can send its
packets to the QP number provided in the arp reply, so it would be
talking not with the rdma cm consumer but rather with the underlying
IPoIB in this node.

On the other direction you are correct, IPoIB does not listen for SIDR requests.

> After the user joins the multicast group, unicast traffic is still supported.

no! it is broken since the PS_IPOIB ID/QP that joined/attached the
multicast group is now using the ipoib broadcast qkey where the PS_UDP
ID/QP is using the RDMA_UDP_QKEY

> The issue I see is whether the rdma_cm uses address resolution (which ends up
> being IP ARP), an SA query, and SIDR to resolve the remote QPN, or if it can
> obtain it through some other method.

A possible fallback to RDMA CM consumer is: issue ARP, then send SIDR
- if there is no response use the IPoIB  QP from the ARP reply and the
ipv4 broadcast qkey to talk directly with IPoIB. However, as i mention
above this hack is not possible in the other direction, that is you
can't make IPoIB do unicast talking with PS_IPOIB consumer.

Or.