[openib-general] FMR and how they work

[Dror Goldenberg]

> -----Original Message-----
> From: Fab Tillier [mailto:ftillier at infiniconsys.com] 
> Sent: Tuesday, May 03, 2005 12:59 AM

> 
> Does the HW2SW_MPT command flush the MTTs referenced by the 
> MPT (if any)? That is, if an MPT has mtt_seg_adr_h and 
> mtt_seg_adr_l set during HW2SW_MPT, do the MTTs get flushed?

Yes.

> 
> The PRM says they're not bindable but doesn't explain why.  
> What happens if an FMR is marked as bindable?

The driver is not supposed to set this bit for proper operation
of the device.

> 
> No, I'm thinking about improving the "slow" registration 
> path.  If I could avoid using the WRITE_MTT command, this 
> would save a few commands (and potential error cases) from 
> the registration path.  I'm envisioning a case where the MTTs 
> are written via a posted write to memory, and then the 
> SW2HW_MPT command is issued for an MPT that references the 
> just-written MTTs.

You can do that, almost as you described. Think of the MR as 
a one shot FMRs. Just go ahead and implement according to 
FMR way such an MR and just use it once.

> 
> Currently (assuming 64MTTs per region), registration is a 2 
> command process for buffers up to 256K.  It becomes 3 
> commands for registrations to 512K, 4 to 768K, and 5 for 1M, 
> and so forth.  Having all of these be just a single SW2HW_MPT 
> command would be nice.
> 
> Doing this would require the MTTs associated with an MPT to 
> get flushed when the HW2SW_MPT command is invoked so that 
> they can be reused in a subsequent registration without 
> hitting stale cached values.

They will get flushed.

> 
> This is also why I'm asking about the bindable properties.  
> Regular memory regions need to be bindable.  Can I do posted 
> writes for the MTTs and use the SW2HW_MPT and HW2SW_MPT 
> commands and retain the bindable property of the MPT?

This is not true. Not all MRs should be bindable. According to IB verbs
spec, at creation time, you decide if an MR is bindable or not. For sure
you can implement your acceleration if the user chose not to enable bind
on the MR. A lot of apps today just don't use binding, so this can be 
a nice assumption for performance improvement.

-Dror
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050503/bb411b7d/attachment.html>