[openib-general] FMR and how they work

[Dror Goldenberg]

> -----Original Message-----
> From: Fab Tillier [mailto:ftillier at infiniconsys.com] 
> Sent: Monday, May 02, 2005 4:50 AM
> 
> Is the HW2SW_MPT command more expensive than the SYNC_TPT 
> command?  

SYNC_TPT wipes out the caches. HW2SW_MPT cleans caches for the
specific MPT. You're welcome to compare the two execution time. I believe
that the main impact on CPU is by the fact that you need to submit a 
command and wait for it to complete (interrupt->EQE).

> Does it just flush the MPT out of the cache?  What 
> happens to that MPT's MTTs - do they get flushed out too 
> (assuming the MPT still references them)?

For correctness you'd have to flush MPTs and MTTs.

> 
> Why aren't FMR's bindable?  

They are not. Check out the PRM.

> It seems that if the HW2SW_MPT 
> flushes the MPT out of the cache, then one could use it for 
> normal memory registrations and avoid the WRITE_MTT command, 
> no?  

Please explain. Maybe you meant to use HW2SW_MPT for Deregistration.
If this is what you mean, then you're right. But it is against the spirit of
"lazy deregistration".  This will only speed the IO operations halfway.
On the creation of mapping it'll be fast - just posted writes to the
MPT/MTT,
on the destruction of the mapping, it'll be slow - because you need 
HW2SW_MPT and you'd need probably to create a new blank MR
for the next reuse.
The intent was to use bulk deregistrations...

> That is, during registration, do posted writes for the 
> MTTs and then a SW2HW_MPT command for an MPT entry that 
> references those MTTs.  Would this work?
> 
> How much slower are memory windows compared to FMRs (assuming 
> the underlying MR is already registered)?

You're welcomed to measure. I think that FMRs will be the fastest way
to create mapping. Faster than MWs.
Also note that FMRs are not bindable. So, there are certain applications
that can benefit from FMRs (e.g. SRP), but can not leverage MWs, at least
I can't think of a model for that...

> 
> Thanks,
> 
> - Fab
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050502/e8f14d44/attachment.html>