[openib-general] How to support IOMMUs for ipath driver

[Or Gerlitz]

Ralph Campbell wrote:
> On Wed, 2006-09-13 at 12:00 +0300, Or Gerlitz wrote:
>> Ralph Campbell wrote:

> Well, the other parts of the kernel might not need a kernel virtual
> address but the ib_ipath driver still does.

So you agree there is a need to kmap/kunamp pages which the user wants 
to  use with IB and are not mapped into the kernel virt address space?

> I don't understand what you are talking about. There is an IB
> wire protocol for RDMA, SEND, etc. That doesn't change depending
> on the HCA.
> The InfiniPath HCA has a ring buffer of receive buffers and all
> incoming IB packets are DMA'ed into one of these buffers.
> The ib_ipath software driver examines the packet and
> copies it to the appropriate address. For a packet received with
> a RC_RDMA_WRITE_FIRST, the RKEY and IB address are used to convert
> that into a kernel virtual address and the data is copied.
> The same happens for RC_SEND_FIRST but the KV address comes from
> the LKEY and address in the work request posted by ib_post_recv().

OK, this make sense.

Lets see if i follow: you say that the Infinipath HCA is RX DMA-able but 
it does RX DMA to the ipath driver private RX buffers and then the 
driver copies from these buffers to the user buffer. My guess is that 
you do that to support both recv and rdma read on this QP since if you 
would only need to support recv you can have the hca dma-ing to the user 
posted rx buffer.

> Sending data is similar, the driver constructs a packet with the
> appropriate opcode and writes it to the chip which puts it on
> the wire.

OK.