[openib-general] High memory

[Shuang Liang]

   Thanks for your reply!
   I am trying to write a gen2 based kernel module here. Basically, what 
I am trying to do is to send a one byte kernel buffer to a remote user 
process. I used get_dma_mr to get the mr before hand at the sender side, 
so that I don't have to bother to register when sending the kernel 
buffer. Since I am working on IA-32 machine, so I assume bus addr and 
phys addr are the same(I tried both virt_to_phys and dma_map_single() 
for addresss translation actually).
   What I noticed was that if the buffer is above in high memory, the 
data was not received correctly; while if in low memory, things work. 
Does this mean HCA is not able to address high memory or maybe I am 
missing sth here?

Thanks
Shuang,
 
Caitlin Bestler wrote:

>openib-general-bounces at openib.org wrote:
>  
>
>>On Thu, Nov 17, 2005 at 12:33:22AM -0500, Shuang Liang wrote:
>>    
>>
>>>Hi,
>>>  I am new here with some problem of gen2 programming, hope somebody
>>>can help me. I was trying to send a message from a kernel buffer to a
>>>remote userland program on IA-32 machines.
>>>  Basically, what happened was I used get_dma_mr to get memory
>>>registered. And I noticed if a buffer is allocated from high memory
>>>(address >f8000000), then the data can not be delivered correctly to
>>>the receiver side(both send recv completes successfully, but with
>>>wrong data). I thought the problem could have been that I used
>>>virt_to_phys for address translation.
>>>      
>>>
>>Using "virt_to_phys" is always wrong when trying to get a DMA mapping.
>>
>>    
>>
>>>But I can't find any appropriate ones for high memory address
>>>translation. I wondering if somebody could give me some suggestions
>>>on this.
>>>      
>>>
>>I don't know enough context.
>>Are you trying to write an openib kernel driver?
>>If so you want to read "Documentation/DMA-API.txt" for hints
>>on the available DMA mapping interfaces and then look at how
>>SDP or IPoIB use the described DMA interfaces.
>>
>>    
>>
>
>That applies even if you are trying to write a kernel daemon
>and wish to use high memory. You either make it part of your
>virtual memory map at least temporarily, so you can register
>it, or you register the memory "physically". But "physical"
>registration for RDMA is never really physical addresses, it
>is always bus addresses, which means reading the stuff meant
>for driver developers.
>
>
>
>  
>


-- 
Shuang Liang, 
Graduate Administration Assistant, 
Department of Computer Science & Engineering
The Ohio State University, 374 Dreese Labs, 2015 Neil Ave
Columbus, Ohio 43210  
614-292-1900, 
614-292-2911 (fax)