[openib-general] High memory

[Caitlin Bestler]

openib-general-bounces at openib.org wrote:
> On Thu, Nov 17, 2005 at 12:33:22AM -0500, Shuang Liang wrote:
>> Hi,
>>   I am new here with some problem of gen2 programming, hope somebody
>> can help me. I was trying to send a message from a kernel buffer to a
>> remote userland program on IA-32 machines.
>>   Basically, what happened was I used get_dma_mr to get memory
>> registered. And I noticed if a buffer is allocated from high memory
>> (address >f8000000), then the data can not be delivered correctly to
>> the receiver side(both send recv completes successfully, but with
>> wrong data). I thought the problem could have been that I used
>> virt_to_phys for address translation.
> 
> Using "virt_to_phys" is always wrong when trying to get a DMA mapping.
> 
>> But I can't find any appropriate ones for high memory address
>> translation. I wondering if somebody could give me some suggestions
>> on this.
> 
> I don't know enough context.
> Are you trying to write an openib kernel driver?
> If so you want to read "Documentation/DMA-API.txt" for hints
> on the available DMA mapping interfaces and then look at how
> SDP or IPoIB use the described DMA interfaces.
> 

That applies even if you are trying to write a kernel daemon
and wish to use high memory. You either make it part of your
virtual memory map at least temporarily, so you can register
it, or you register the memory "physically". But "physical"
registration for RDMA is never really physical addresses, it
is always bus addresses, which means reading the stuff meant
for driver developers.