[openib-general] mthca FMR correctness (and memory windows)

[Talpey, Thomas]

I'm implementing FMR memory registration mode in the NFS/RDMA
client, and I've got it mostly working. However as I understand it,
mthca's existing fmr's do not guarantee that the r_key is completely
invalidated when the ib_unmap_fmr() returns. This makes using them
rather problematic, to say the least.

Now, I notice that ib_unmap_fmr() is a blocking operation (at least,
the kernel whines about semaphores being waited in interrupt context,
when I experimented with that).

Does this mean mthca's ib_unmap_fmr() is waiting for the invalidation
now, or plans to in the future?

Second comment is that the existing fmr api is (IMO) very inconsistent.
Why does ib_map_phys_fmr() take an array of u64 physaddrs and not
struct page *'s? And the unmap api mysteriously takes a struct list *,
not any object returned by ib_alloc_fmr() or ib_map_phys_fmr().

Is there a plan to make fmr's compliant with verbs 1.2?

Final question is memory windows. The code is there in the NFS/RDMA
client, but all it gets is ENOSYS from ib_bind_mw(). (Yes, I know memory
windows may not perform well on mthca. However, they're correct and
hopefully faster than ib_reg_phys_mr()).

What is the plan to implement mthca memory windows?

Thanks,
Tom.