[openib-general] [PATCH 1/7] IB/core - Add DMA mapping functions to allow device drivers to interpose

[Michael S. Tsirkin]

Quoting r. Ralph Campbell <ralph.campbell at qlogic.com>:
> Subject: [PATCH 1/7] IB/core - Add DMA mapping functions to allow device drivers to interpose
> 
> IB/core - Add DMA mapping functions to allow device drivers to interpose
> 
> The QLogic InfiniPath HCAs use programmed I/O instead of HW DMA.
> This patch allows a verbs device driver to interpose on DMA mapping
> function calls in order to avoid relying on bus_to_virt() and
> phys_to_virt() to undo the mappings created by dma_map_single(),
> dma_map_sg(), etc.
> 
> From: Ralph Campbell <ralph.campbell at qlogic.com>
> 
> diff -r f37bd0e41fec include/rdma/ib_verbs.h
> --- a/include/rdma/ib_verbs.h	Thu Oct 26 21:44:41 2006 +0700
> +++ b/include/rdma/ib_verbs.h	Thu Oct 26 16:10:04 2006 -0800
> @@ -43,6 +43,8 @@
>  
>  #include <linux/types.h>
>  #include <linux/device.h>
> +#include <linux/mm.h>
> +#include <linux/dma-mapping.h>
>  
>  #include <asm/atomic.h>
>  #include <asm/scatterlist.h>
> @@ -846,6 +848,42 @@ struct ib_cache {
>  	struct ib_pkey_cache  **pkey_cache;
>  	struct ib_gid_cache   **gid_cache;
>  	u8                     *lmc_cache;
> +};
> +
> +struct ib_dma_mapping_ops {
> +	int		(*mapping_error)(struct ib_device *dev,
> +					 dma_addr_t dma_addr);
> +	dma_addr_t	(*map_single)(struct ib_device *dev,
> +				      void *ptr, size_t size,
> +				      enum dma_data_direction direction);
> +	void		(*unmap_single)(struct ib_device *dev,
> +					dma_addr_t addr, size_t size,
> +					enum dma_data_direction direction);
> +	dma_addr_t	(*map_page)(struct ib_device *dev,
> +				    struct page *page, unsigned long offset,
> +				    size_t size,
> +				    enum dma_data_direction direction);
> +	void		(*unmap_page)(struct ib_device *dev,
> +				      dma_addr_t addr, size_t size,
> +				      enum dma_data_direction direction);
> +	int		(*map_sg)(struct ib_device *dev,
> +				  struct scatterlist *sg, int nents,
> +				  enum dma_data_direction direction);
> +	void		(*unmap_sg)(struct ib_device *dev,
> +				    struct scatterlist *sg, int nents,
> +				    enum dma_data_direction direction);
> +	dma_addr_t	(*dma_address)(struct ib_device *dev,
> +				       struct scatterlist *sg);
> +	unsigned int	(*dma_len)(struct ib_device *dev,
> +				   struct scatterlist *sg);
> +	void		(*sync_single_for_cpu)(struct ib_device *dev,
> +					       dma_addr_t dma_handle,
> +					       size_t size,
> +				               enum dma_data_direction dir);
> +	void		(*sync_single_for_device)(struct ib_device *dev,
> +						  dma_addr_t dma_handle,
> +						  size_t size,
> +						  enum dma_data_direction dir);
>  };

Maybe we should make the API a bit more generic than just matching what ipath
needs. Specifically mellanox HCAs (and I expect others) can support *both* dma
(in use today) and pushing data "inline" directly into HCA.

And this actually might make more sense than DMA for small messages.

However, this means that the API must give the HCA the choice of
what to keep inside the mapping. This could mean, for example, returning
a structure that can include dma_addr_t, void*, or both, and a flag to
distinguish between the two.

Does this make sense?

-- 
MST