[openib-general] mthca FMR correctness (and memory windows)

[Talpey, Thomas]

At 02:36 PM 3/20/2006, Dror Goldenberg wrote:
>If you use the pool, you can just return the fmr to the pool instead
>of explicitly deregistering it. Deregistration is deferred and done in 
>batch by a helper thread that belongs to the pool. Fmrs remapping 
>can also utilize a cache that can speed performance.

Uh, no can do! When the RPC layer indicates to NFS that the
reply has been received, it must ensure that the reply buffers
are synchronized and stable, i.e. the rkey is invalidated. For the
same reason, it's unacceptable to share rkey's, which appears
to be another goal of the fmr pool.

So, for now I'll definitely stick to the direct fmr map/unmap in the
RPC path.

>I didn't say nobody... I was saying that so far there wasn't any demand
>for them, at least not on openib mailing list. Anyway, the HW supports 
>[memory windows, just need to write the piece of code in mthca that will create the
>WQE and post it.

Hmm - okay I might try that! Looking at the code, it almost looks
as if I carefully build a bind WR, I could maybe just use ib_post_send()
to post it... ?

Thanks,
Tom.