[openib-general] FMR and how they work

[Fab Tillier]

> From: Dror Goldenberg [mailto:gdror at mellanox.co.il]
> Sent: Sunday, May 01, 2005 6:12 AM
> 
> > -----Original Message-----
> > From: Fab Tillier [mailto:ftillier at infiniconsys.com]
> > Sent: Thursday, April 28, 2005 6:00 PM
> >
> > Can an incoming RDMA
> > transfer access the pages previously referenced by that FMR?
> > If so, for how long?  When is it safe to unpin the pages?
>
> As long as the entries are cached, then the answer is yes. Obviously
> according to the FMR access rights and PD which are always validated.
> The entries get out of the cache either when cache is trashed or when
> you explicitly flush it. If you want explicit cache flush, then you can
> use SYNC_TPT  command.
> Destruction of the object (HW2SW_MPT) also updates the caches
> but is an expensive operation.
>

Is the HW2SW_MPT command more expensive than the SYNC_TPT command?  Does it
just flush the MPT out of the cache?  What happens to that MPT's MTTs - do
they get flushed out too (assuming the MPT still references them)?

Why aren't FMR's bindable?  It seems that if the HW2SW_MPT flushes the MPT
out of the cache, then one could use it for normal memory registrations and
avoid the WRITE_MTT command, no?  That is, during registration, do posted
writes for the MTTs and then a SW2HW_MPT command for an MPT entry that
references those MTTs.  Would this work?

How much slower are memory windows compared to FMRs (assuming the underlying
MR is already registered)?

Thanks,

- Fab