[openib-general] mthca FMR correctness (and memory windows)

[Dror Goldenberg]

> From: Talpey, Thomas [mailto:Thomas.Talpey at netapp.com] 
> Sent: Monday, March 20, 2006 5:11 PM
> 

> 
> Currently I must defer the deregistration until processing 
> reaches user context, and then the blocking operation costs a 
> context switch. With the memory window API, I can launch the 
> deregistration early, and it's often polled as complete by 
> the time I'm ready to return from the RPC. 
> So, I would prefer that fmr's used a similar process.

If you use the pool, you can just return the fmr to the pool instead
of explicitly deregistering it. Deregistration is deferred and done in 
batch by a helper thread that belongs to the pool. Fmrs remapping 
can also utilize a cache that can speed performance.

> I will not agree with your statement that nobody wants memory 
> windows. User space applications that don't wish to expose 
> large amounts of memory will certainly want them. Kernel 
> space has the advantage here, by being able to use fmr. User 
> space can't do that.

I didn't say nobody... I was saying that so far there wasn't any demand
for them, at least not on openib mailing list. Anyway, the HW supports 
them, just need to write the piece of code in mthca that will create the

WQE and post it.