<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2654.45">
<TITLE>RE: [openib-general] FMR and how they work</TITLE>
</HEAD>
<BODY>
<BR>
<BR>
<P><FONT SIZE=2>> -----Original Message-----</FONT>
<BR><FONT SIZE=2>> From: Fab Tillier [<A HREF="mailto:ftillier@infiniconsys.com">mailto:ftillier@infiniconsys.com</A>] </FONT>
<BR><FONT SIZE=2>> Sent: Tuesday, May 03, 2005 12:59 AM</FONT>
</P>
<P><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Does the HW2SW_MPT command flush the MTTs referenced by the </FONT>
<BR><FONT SIZE=2>> MPT (if any)? That is, if an MPT has mtt_seg_adr_h and </FONT>
<BR><FONT SIZE=2>> mtt_seg_adr_l set during HW2SW_MPT, do the MTTs get flushed?</FONT>
</P>
<P><FONT SIZE=2>Yes.</FONT>
</P>
<P><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> The PRM says they're not bindable but doesn't explain why. </FONT>
<BR><FONT SIZE=2>> What happens if an FMR is marked as bindable?</FONT>
</P>
<P><FONT SIZE=2>The driver is not supposed to set this bit for proper operation</FONT>
<BR><FONT SIZE=2>of the device.</FONT>
</P>
<P><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> No, I'm thinking about improving the "slow" registration </FONT>
<BR><FONT SIZE=2>> path. If I could avoid using the WRITE_MTT command, this </FONT>
<BR><FONT SIZE=2>> would save a few commands (and potential error cases) from </FONT>
<BR><FONT SIZE=2>> the registration path. I'm envisioning a case where the MTTs </FONT>
<BR><FONT SIZE=2>> are written via a posted write to memory, and then the </FONT>
<BR><FONT SIZE=2>> SW2HW_MPT command is issued for an MPT that references the </FONT>
<BR><FONT SIZE=2>> just-written MTTs.</FONT>
</P>
<P><FONT SIZE=2>You can do that, almost as you described. Think of the MR as </FONT>
<BR><FONT SIZE=2>a one shot FMRs. Just go ahead and implement according to </FONT>
<BR><FONT SIZE=2>FMR way such an MR and just use it once.</FONT>
</P>
<P><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Currently (assuming 64MTTs per region), registration is a 2 </FONT>
<BR><FONT SIZE=2>> command process for buffers up to 256K. It becomes 3 </FONT>
<BR><FONT SIZE=2>> commands for registrations to 512K, 4 to 768K, and 5 for 1M, </FONT>
<BR><FONT SIZE=2>> and so forth. Having all of these be just a single SW2HW_MPT </FONT>
<BR><FONT SIZE=2>> command would be nice.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Doing this would require the MTTs associated with an MPT to </FONT>
<BR><FONT SIZE=2>> get flushed when the HW2SW_MPT command is invoked so that </FONT>
<BR><FONT SIZE=2>> they can be reused in a subsequent registration without </FONT>
<BR><FONT SIZE=2>> hitting stale cached values.</FONT>
</P>
<P><FONT SIZE=2>They will get flushed.</FONT>
</P>
<P><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> This is also why I'm asking about the bindable properties. </FONT>
<BR><FONT SIZE=2>> Regular memory regions need to be bindable. Can I do posted </FONT>
<BR><FONT SIZE=2>> writes for the MTTs and use the SW2HW_MPT and HW2SW_MPT </FONT>
<BR><FONT SIZE=2>> commands and retain the bindable property of the MPT?</FONT>
</P>
<P><FONT SIZE=2>This is not true. Not all MRs should be bindable. According to IB verbs</FONT>
<BR><FONT SIZE=2>spec, at creation time, you decide if an MR is bindable or not. For sure</FONT>
<BR><FONT SIZE=2>you can implement your acceleration if the user chose not to enable bind</FONT>
<BR><FONT SIZE=2>on the MR. A lot of apps today just don't use binding, so this can be </FONT>
<BR><FONT SIZE=2>a nice assumption for performance improvement.</FONT>
</P>
<P><FONT SIZE=2>-Dror</FONT>
</P>
</BODY>
</HTML>