[ofa-general] [PATCH RFC] ib_mthca: avoid recycling FMR R_Keys too soon

[Or Gerlitz]

Jack Morgenstein wrote:
> On Thursday 21 February 2008 13:42, Or Gerlitz wrote:
>> As far as I understand under Sinai you must issue an adjust_key call 
>> when the key is about to wraparound, correct?

> Actually, its not related to wraparound.  The key adjustment is in the
> mpt-index section only, and does not affect the sequence number section.
> If we don't re-initialize the key, adjust_key should not be called.

Is it possible to never re-initialize the key? if yes, what's the 
semantics of the M=max_map_per_fmr device attribute? I was thinking that 
after the fmr was mapped M times, something --has-- to be reinitialized, 
sorry if this is my misunderstanding, can you clarify that?

Or

> commit d4cb0784fd1ea99ef3d20526811bd5608146fe60
> Author: Or Gerlitz <ogerlitz at voltaire.com>
> Date:   Sat Jun 17 20:37:37 2006 -0700
> 
>     IB/mthca: Fill in max_map_per_fmr device attribute
> 
>     Report the true max_map_per_fmr value from mthca_query_device(),
>     taking into account the change in FMR remapping introduced by the
>     Sinai performance optimization.
> 
>     Signed-off-by: Or Gerlitz <ogerlitz at voltaire.com>
>     Signed-off-by: Roland Dreier <rolandd at cisco.com>
> 
> diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c
> index a2eae8a..8f89ba7 100644
> --- a/drivers/infiniband/hw/mthca/mthca_provider.c
> +++ b/drivers/infiniband/hw/mthca/mthca_provider.c
> @@ -115,6 +115,16 @@ static int mthca_query_device(struct ib_device *ibdev,
>         props->max_mcast_qp_attach = MTHCA_QP_PER_MGM;
>         props->max_total_mcast_qp_attach = props->max_mcast_qp_attach *
>                                            props->max_mcast_grp;
> +       /*
> +        * If Sinai memory key optimization is being used, then only
> +        * the 8-bit key portion will change.  For other HCAs, the
> +        * unused index bits will also be used for FMR remapping.
> +        */
> +       if (mdev->mthca_flags & MTHCA_FLAG_SINAI_OPT)
> +               props->max_map_per_fmr = 255;
> +       else
> +               props->max_map_per_fmr =
> +                       (1 << (32 - long_log2(mdev->limits.num_mpts))) - 1;
> 
>         err = 0;
>   out: