<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; ">Tziporet, <DIV><BR class="khtml-block-placeholder"></DIV><DIV>This did appear to be a firmware issue. After the update I have a latency difference on the order of .2 uSec which seems reasonable, much better than a ~5. uSec penalty! </DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Thanks for the help, </DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Galen </DIV><DIV> </DIV><DIV><BR><DIV><DIV>On Jul 27, 2005, at 12:38 AM, Tziporet Koren wrote:</DIV><BR class="Apple-interchange-newline"><BLOCKQUOTE type="cite"> <DIV><FONT color="#0000ff" face="Arial" size="2"><SPAN class="562031806-27072005">Hi Galen,</SPAN></FONT></DIV> <DIV><FONT color="#0000ff" face="Arial" size="2"><SPAN class="562031806-27072005">You are working with an old FW version that had issues with SRQ performance.</SPAN></FONT></DIV> <DIV><FONT color="#0000ff" face="Arial" size="2"><SPAN class="562031806-27072005">We did performance improvements and now the latency and BW with SRQ are the same as with regular QP.</SPAN></FONT></DIV> <DIV><FONT color="#0000ff" face="Arial" size="2"><SPAN class="562031806-27072005"></SPAN></FONT> </DIV> <DIV><FONT color="#0000ff" face="Arial" size="2"><SPAN class="562031806-27072005">The changes are available in the new FW release we did few weeks ago: see <A href="http://www.mellanox.com/products/firmware.html">http://www.mellanox.com/products/firmware.html</A></SPAN></FONT></DIV> <DIV><FONT color="#0000ff" face="Arial" size="2"><SPAN class="562031806-27072005"></SPAN></FONT> </DIV> <DIV><FONT color="#0000ff" face="Arial" size="2"><SPAN class="562031806-27072005">Regarding your specific device: you are using </SPAN></FONT><SPAN class="562031806-27072005"><FONT size="2"><FONT color="#0000ff"><FONT face="Arial"><SPAN class="931331910-15062005">MT25208<SPAN class="562031806-27072005"> </SPAN>InfiniHost III Ex, so you need to use FW 4.7.0</SPAN></FONT></FONT></FONT></SPAN></DIV> <DIV><FONT size="2"><FONT color="#0000ff"><FONT face="Arial"> <DIV> <SPAN class="562031806-27072005"> </SPAN></DIV> <DIV><SPAN class="562031806-27072005">In case you need help with FW burning please approach our FAE Todd (cc here too).</SPAN></DIV> <DIV><SPAN class="562031806-27072005"></SPAN> </DIV> <DIV><SPAN class="562031806-27072005">After you have new results please reply if the performance numbers are OK now.</SPAN></DIV> <DIV><SPAN class="562031806-27072005"></SPAN> </DIV> <DIV><SPAN class="562031806-27072005">Thanks,</SPAN></DIV> <DIV><SPAN class="562031806-27072005">Tziporet</SPAN></DIV> <DIV><SPAN class="562031806-27072005"></SPAN> </DIV> </FONT></FONT><DIV><FONT color="#0000ff"><FONT face="Arial"><SPAN class="562031806-27072005"> </SPAN></FONT></FONT><FONT face="Tahoma"></FONT><FONT size="2">-----Original Message-----<BR><B>From:</B> Galen M. Shipman [<A href="mailto:gshipman@lanl.gov">mailto:gshipman@lanl.gov</A>]<BR><B>Sent:</B> Tuesday, July 26, 2005 5:55 PM<BR><B>To:</B> Tziporet Koren<BR><B>Subject:</B> Re: [openib-general] Shared Receive Queue<BR><BR></FONT></DIV><FONT size="2"></FONT></FONT></DIV><FONT size="2"> </FONT><BLOCKQUOTE><FONT size="2"></FONT> <DIV>The following is from vstat.. </DIV> <DIV><BR class="khtml-block-placeholder"></DIV> <DIV> hca_id=InfiniHost0</DIV> <DIV> vendor_id=0x02C9</DIV> <DIV> vendor_part_id=0x6278</DIV> <DIV> hw_ver=0xA0</DIV> <DIV> fw_ver=0x400050003</DIV> <DIV> num_phys_ports=2</DIV> <DIV> port=1</DIV> <DIV> port_state=PORT_DOWN</DIV> <DIV> sm_lid=0x0000</DIV> <DIV> port_lid=0x0000</DIV> <DIV> port_lmc=0x00</DIV> <DIV> max_mtu=2048</DIV> <DIV><BR class="khtml-block-placeholder"></DIV> <DIV> port=2</DIV> <DIV> port_state=PORT_ACTIVE</DIV> <DIV> sm_lid=0x0001</DIV> <DIV> port_lid=0x010a</DIV> <DIV> port_lmc=0x00</DIV> <DIV> max_mtu=2048</DIV> <DIV><BR class="khtml-block-placeholder"></DIV><BR> <DIV> <DIV>On Jul 26, 2005, at 8:46 AM, Tziporet Koren wrote:</DIV><BR class="Apple-interchange-newline"> <BLOCKQUOTE type="cite"><P><FONT size="2">which HW & FW are you using?</FONT> </P><P><FONT size="2">-----Original Message-----</FONT> <BR><FONT size="2">From: Roland Dreier [<A href="mailto:rolandd@cisco.com">mailto:rolandd@cisco.com</A>]</FONT> <BR><FONT size="2">Sent: Tuesday, July 26, 2005 5:38 PM</FONT> <BR><FONT size="2">To: Galen M. Shipman</FONT> <BR><FONT size="2">Cc: <A href="mailto:openib-general@openib.org">openib-general@openib.org</A></FONT> <BR><FONT size="2">Subject: Re: [openib-general] Shared Receive Queue</FONT> </P><BR><P><FONT size="2"> Galen> We have software which uses the mellanox shared receive</FONT> <BR><FONT size="2"> Galen> queue and we are seeing a substantial performance penalty</FONT> <BR><FONT size="2"> Galen> (on the order of 4-5 usec on 0 byte messages). I don't know</FONT> <BR><FONT size="2"> Galen> all the technical details but if you could keep this in</FONT> <BR><FONT size="2"> Galen> mind when working on the shared receive queue perhaps the</FONT> <BR><FONT size="2"> Galen> performance penalty could be much less in gen2.</FONT> </P><P><FONT size="2">Is this performance penalty in the software? It seems much more</FONT> <BR><FONT size="2">likely that the issue is in the hardware implementation of SRQ.</FONT> </P><P><FONT size="2"> - R.</FONT> <BR><FONT size="2">_______________________________________________</FONT> <BR><FONT size="2">openib-general mailing list</FONT> <BR><FONT size="2"><A href="mailto:openib-general@openib.org">openib-general@openib.org</A></FONT> <BR><FONT size="2"><A href="http://openib.org/mailman/listinfo/openib-general" target="_blank">http://openib.org/mailman/listinfo/openib-general</A></FONT> </P><P><FONT size="2">To unsubscribe, please visit <A href="http://openib.org/mailman/listinfo/openib-general" target="_blank">http://openib.org/mailman/listinfo/openib-general</A></FONT> </P></BLOCKQUOTE></DIV><BR></BLOCKQUOTE></BLOCKQUOTE></DIV><BR></DIV></BODY></HTML>