[ewg] Fwd: [ofa-general] [PATCH/RFC] IPoIB: Handle case when P_Key is deleted and re-added at same index

[Vladimir Sokolovsky]

Olga Shern (Voltaire) wrote:
> Hi Vlad,
>  
> Please add this patch to OFED 1.3.1
> In additional to the main purpose of this patch it is also fixes issues 
> we saw with partitioning and SM failover because of:
>  
> /Also, switch to using ib_find_pkey() instead of ib_find_cached_pkey()
> everywhere in IPoIB, since none of the places that look for P_Keys are
> in a fast path or in non-sleeping context, and in general we want to
> kill off the whole caching infrastructure eventually.  This also fixes
> consistency problems caused because some IPoIB queries were cached and
> some were uncached during the window where the cache was not updated./
> // 
> Thanks
> Olga
> 
> 
> ---------- Forwarded message ----------
> From: *Roland Dreier* <rdreier at cisco.com <mailto:rdreier at cisco.com>>
> Date: Apr 15, 2008 8:55 AM
> Subject: [ofa-general] [PATCH/RFC] IPoIB: Handle case when P_Key is 
> deleted and re-added at same index
> To: general at lists.openfabrics.org <mailto:general at lists.openfabrics.org>
> 
> If a P_Key is deleted and then re-added at the same index, then IPoIB
> gets confused because __ipoib_ib_dev_flush() only checks whether the
> index is the same without checking whether the P_Key was present, so
> the interface is stopped when the P_Key is deleted, but the event when
> the P_Key is re-added gets ignored and the interface never gets
> restarted.
> 
> Also, switch to using ib_find_pkey() instead of ib_find_cached_pkey()
> everywhere in IPoIB, since none of the places that look for P_Keys are
> in a fast path or in non-sleeping context, and in general we want to
> kill off the whole caching infrastructure eventually.  This also fixes
> consistency problems caused because some IPoIB queries were cached and
> some were uncached during the window where the cache was not updated.
> 
> Thanks to Venkata Subramonyam <vsubramo at cisco.com 
> <mailto:vsubramo at cisco.com>> for debugging this
> problem and testing this fix.
> 
> Signed-off-by: Roland Dreier <rolandd at cisco.com <mailto:rolandd at cisco.com>>
> ---
> drivers/infiniband/ulp/ipoib/ipoib_cm.c |    4 ++--
> drivers/infiniband/ulp/ipoib/ipoib_ib.c |   10 +++++-----
> 2 files changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c 
> b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
> index 9d411f2..9db7b0b 100644
> --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
> +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
> @@ -1007,9 +1007,9 @@ static int ipoib_cm_modify_tx_init(struct 
> net_device *dev,
>        struct ipoib_dev_priv *priv = netdev_priv(dev);
>        struct ib_qp_attr qp_attr;
>        int qp_attr_mask, ret;
> -       ret = ib_find_cached_pkey(priv->ca, priv->port, priv->pkey, 
> &qp_attr.pkey_index);
> +       ret = ib_find_pkey(priv->ca, priv->port, priv->pkey, 
> &qp_attr.pkey_index);
>        if (ret) {
> -               ipoib_warn(priv, "pkey 0x%x not in cache: %d\n", 
> priv->pkey, ret);
> +               ipoib_warn(priv, "pkey 0x%x not found: %d\n", 
> priv->pkey, ret);
>                return ret;
>        }
> 
> diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c 
> b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> index 8b4ff69..0205eb7 100644
> --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> @@ -594,7 +594,7 @@ static void ipoib_pkey_dev_check_presence(struct 
> net_device *dev)
>        struct ipoib_dev_priv *priv = netdev_priv(dev);
>        u16 pkey_index = 0;
> 
> -       if (ib_find_cached_pkey(priv->ca, priv->port, priv->pkey, 
> &pkey_index))
> +       if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &pkey_index))
>                clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags);
>        else
>                set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags);
> @@ -835,13 +835,13 @@ static void __ipoib_ib_dev_flush(struct 
> ipoib_dev_priv *priv, int pkey_event)
>                        clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags);
>                        ipoib_ib_dev_down(dev, 0);
>                        ipoib_ib_dev_stop(dev, 0);
> -                       ipoib_pkey_dev_delay_open(dev);
> -                       return;
> +                       if (ipoib_pkey_dev_delay_open(dev))
> +                               return;
>                }
> -               set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags);
> 
>                /* restart QP only if P_Key index is changed */
> -               if (new_index == priv->pkey_index) {
> +               if (test_and_set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags) &&
> +                   new_index == priv->pkey_index) {
>                        ipoib_dbg(priv, "Not flushing - P_Key index not 
> changed.\n");
>                        return;
>                }
> --
> 1.5.5
>


Hello Olga,
This patch can't be applied as is to the ofed-1.3.1 git tree:

patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c
Hunk #1 succeeded at 847 (offset -160 lines).
patching file drivers/infiniband/ulp/ipoib/ipoib_ib.c
Hunk #1 succeeded at 488 (offset -106 lines).
Hunk #2 FAILED at 729.
1 out of 2 hunks FAILED -- saving rejects to file drivers/infiniband/ulp/ipoib/ipoib_ib.c.rej

Can you recreate this patch against git://git.openfabrics.org/ofed_1_3/linux-2.6.git ofed_kernel?

Regards,
Vladimir