[openib-general] [kernel verbs] u64 vs dma_addr_t

[Caitlin Bestler]

openib-general-bounces at openib.org wrote:
> sean> James Lentini wrote:
> sean> > Why is the ib_sge's addr a u64 and not a dma_addr_t?
> sean>
> sean> It's the same address that the user can transfer to the remote
> sean> side. 
> 
> It can be the same address, but does it have to be?
> 
> A user can directly map local addresses to InfiniBand I/O
> virtual addresses, but I don't think it is a requirement. In
> other words, I thought that user could register address x and
> request an InfiniBand I/O virtual address of y, x != y, for
> the mapping.
> 
> I understand why the ib_send_wr's rdma.remote_addr needs to
> be a u64, since it ultimately winds up on the wire.
> 
> In the case of the ib_sge's addr, I didn't think these values
> left the local node. My assumption (based on looking at the
> mthca driver) is that they are supposed to contain "local"
> I/O addresses (bus addresses). Therefore, my confusion over
> why dma_addr_t wasn't used.
> 


A privileged user, such as an NFS Daemon or iSER iSCSI Target,
can and will create Memory Regions that are not part of its
own address space out of page buffers. Even running on a 32-bit
kernel it might create a memory region larger than 2**32.

Admittedly, that isn't very likely unless it is the *only*
daemon running on the machine. But it is legal.