[ofa-general] Re: Demand paging for memory regions

[Patrick Geoffray]

Jason Gunthorpe wrote:
> [mangled CC list trimmed]
Thanks, noticed that afterwards.

> This wasn't ment as a slight against Quadrics, only to point out that
> the specific wire protcols used by IB and iwarp are what cause this
> limitation, it would be easy to imagine that Quadrics has some
> additional twist that can make this easier..

The wire protocols are similar, nothing fancy. The specificity of 
Quadrics (and many others) is that they can change the behavior of the 
NIC in firmware, so they adapt to what the OS offers. They had the VM 
notifier support in Tru64 back in the days, they just ported the 
functionality to Linux.

> I ment that HPC users are unlikely to want to swap active RDMA pages
> if this causes a performance cost on normal operations. None of my

Swapping to disk is not a normal operations in HPC, it's going to be 
slow anyway. The main problem for HPC users is not swapping, it's that 
they do not know when a registered page is released to the OS through 
free(), sbrk() or munmap(). Like swapping, they don't expect that it 
will happen often, but they have to handle it gracefully.

Patrick
-- 
Patrick Geoffray
Myricom, Inc.
http://www.myri.com