[openib-general][patch review] srp: fmr implementation,

[Talpey, Thomas]

At 03:12 PM 5/9/2006, Roland Dreier wrote:
>BTW, does Mellanox (or anyone else) have any numbers showing that
>using FMRs makes any difference in performance on a semi-realistic benchmark?

Not me. Using the current FMRs to register/deregister windows for
each NFS/RDMA operation yields only a slight performance improvement
over ib_reg_phys_mr(), and I suspect this is mainly from the fact that
FMRs are page-rounded. Additionally, I find that the queuepair (or perhaps
the completion queue) seems to hang unpredictably, new events get
stuck, only to flush after the upper layer times out and closes the
connection.

What I really don't like about the current FMRs is that they seem to
be optimized only for lazy-deregistration, the fmr pools attempt to defer
the deregistration somewhat indefinitely. This is an enormous security
hole, and pretty much defeats the point of dynamic registration. The
NFS/RDMA client has full-physical mode for users that want speed
in well-protected environments. And it's a LOT faster.

I am planning to test this some more in the next few weeks, but what
I'd really like to see is an IBTA 1.2-compliant implementation, and one
that operated on work queue entries (not synchronous verbs). Is that
being worked on?

Tom.