[openib-general] [PATCH 1/2] mthca support for max_map_per_fmr device attribute

[Or Gerlitz]

The max fmr remaps device attribute is not set by the driver, so the generic 
fmr_pool uses a default of 32. Enlaring this quantity would make the amortized 
cost of remaps lower. With the current mthca "default profile" on memfull HCA 
17 bits are used for MPT addressing so an FMR can be remapped 2^15 - 1 >> 32 times.

Looking in the VAPI driver, i understand there might be some issue with the 
memfree HCA(s) which can break this patch. In vapi/kernel/mlxhh/rhh/hob.c 
:: XHH_hob_query_struct_init() i see the following code:

+ /* max maps per fmr: in Arbel we use the 8-LSbits for FMR memkey modification */
+ hca_cap_p->max_num_map_per_fmr = 255;

I am not sure if the RHH code is used for both Arbel and Sinai memfree 
or only for Sinai.

Also, if the patch makes sense and the memfree issue is resolved, i'd 
like to change the name of the device attribute from max_map_per_fmr 
to max_remaps_per_fmr, i can resend this patch series with this fix.

Or.

Signed-off-by: Or Gerlitz <ogerlitz at voltaire.com>

Index: hw/mthca/mthca_provider.c
===================================================================
--- hw/mthca/mthca_provider.c	(revision 7031)
+++ hw/mthca/mthca_provider.c	(working copy)
@@ -116,6 +116,11 @@ static int mthca_query_device(struct ib_
 	props->max_total_mcast_qp_attach = props->max_mcast_qp_attach *
 					   props->max_mcast_grp;
 
+	/* FMR can be remapped 2^B - 1 times where B < 32 is the number of bits
+	 *  which are not used for MPT addressing 
+	 */
+	props->max_map_per_fmr	   = (1 << (32 - 
+					long_log2(mdev->limits.num_mpts))) - 1;
 	err = 0;
  out:
 	kfree(in_mad);