[ofa-general] [RFC PATCH 4/4] rdma/cma: implement RDMA_ALIGN_WITH_NETDEVICE ha mode

[Or Gerlitz]

On 5/13/08, Roland Dreier <rdreier at cisco.com> wrote:
>
> > Use netevent notification for sensing that a change has happened in the
> IP stack,
>   > then scan the rdma-cm IDs list to see if there is an ID that is
> "misaligned"
>   > in that respect with the IP stack, and disconnect it, in case this is
> what the
>   > user asked to when setting an ha mode for the ID.
>
> this seems like a strange "HA" feature -- to disconnect connections that
> otherwise would continue operating.  What is the use case/use scenario?


OK, I might went too fast here. The idea is to align the RDMA traffic with
the links used by the IP stack. In the case where the app takes advantage of
bonding ipoib devices to achieve HA AND it want this alignment, when bonding
does fail-over from --any-- reason (eg the problem is fixed and the primary
option is used) a "fail-back" of the connection is needed.

(*) HW error --> RC connection break && bonding failover (change of active
slave device, send gratuitous ARP), then this app reconnects and its back in
the business.


  > +    list_for_each_entry(cma_dev, &dev_list, list)
>   > +            list_for_each_entry(id_priv, &cma_dev->id_list, list) {
>   > +                    dev_addr = &id_priv->id.route.addr.dev_addr;
>   > +                    if (!memcmp(dev_addr->src_netdev_name,
> ndev->name, IFNAMSIZ) &&
>   > +                            memcmp(dev_addr->src_dev_addr,
> ndev->dev_addr, ndev->addr_len))
>   > +                                    if (id_priv->ha_mode ==
> RDMA_ALIGN_WITH_NETDEVICE)
>   >
> +                                            schedule_work(&id_priv->ha_work);
>   > +            }
>
>
> This looks horribly racy/incorrect against RDMA device removal, CMA ID
> destruction and netdev renaming.


mmm, bad. I see your point re the first two, that is some locking is needed
to protect against device removal, the ID should be referenced, etc. As for
the netdev renaming, I don't see how making a decsion based on
memcmp(dev_addr->src_dev_name, ndev->name, IFNAMSIZ) is racy?! even for the
crazy case the ndev->name gets changed in the middle of this memcmp, the
only issue would be some confusion made by the code, no damage.

Or.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080513/25233a27/attachment.html>