[openib-general] mthca FMR correctness (and memory windows)
Thomas.Talpey at netapp.com
Sun Mar 19 07:36:11 PST 2006
I'm implementing FMR memory registration mode in the NFS/RDMA
client, and I've got it mostly working. However as I understand it,
mthca's existing fmr's do not guarantee that the r_key is completely
invalidated when the ib_unmap_fmr() returns. This makes using them
rather problematic, to say the least.
Now, I notice that ib_unmap_fmr() is a blocking operation (at least,
the kernel whines about semaphores being waited in interrupt context,
when I experimented with that).
Does this mean mthca's ib_unmap_fmr() is waiting for the invalidation
now, or plans to in the future?
Second comment is that the existing fmr api is (IMO) very inconsistent.
Why does ib_map_phys_fmr() take an array of u64 physaddrs and not
struct page *'s? And the unmap api mysteriously takes a struct list *,
not any object returned by ib_alloc_fmr() or ib_map_phys_fmr().
Is there a plan to make fmr's compliant with verbs 1.2?
Final question is memory windows. The code is there in the NFS/RDMA
client, but all it gets is ENOSYS from ib_bind_mw(). (Yes, I know memory
windows may not perform well on mthca. However, they're correct and
hopefully faster than ib_reg_phys_mr()).
What is the plan to implement mthca memory windows?
More information about the general