[openib-general] [PATCH for-2.6.21] IPoIB/cm: improve small message bandwidth

[Or Gerlitz]

Michael S. Tsirkin wrote:
> Avoid overhead of freeing/reallocating and mapping/unmapping for dma
> for pages that have not been written to by hardware.

> diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
> index 8ee6f06..a23c8e3 100644
> --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
> +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
> @@ -68,14 +68,14 @@ struct ipoib_cm_id {
>  static int ipoib_cm_tx_handler(struct ib_cm_id *cm_id,
>  			       struct ib_cm_event *event);
>  
> -static void ipoib_cm_dma_unmap_rx(struct ipoib_dev_priv *priv,
> +static void ipoib_cm_dma_unmap_rx(struct ipoib_dev_priv *priv, int frags,
>  				  u64 mapping[IPOIB_CM_RX_SG])
>  {
>  	int i;
>  
>  	ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE);
>  
> -	for (i = 0; i < IPOIB_CM_RX_SG - 1; ++i)
> +	for (i = 0; i < frags; ++i)
>  		ib_dma_unmap_single(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE);
>  }

I understand that in ipoib_cm_alloc_rx_skb you call dma_map_page on 
IPOIB_CM_RX_SG pages where here you call dma_unmap_single only $frags 
times, correct? does this means you are trashing the IOMMU etc etc of 
the system?

Or.