<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2654.45">
<TITLE>RE: [Openib-windows] IPoIB GUID to MAC conversion</TITLE>
</HEAD>
<BODY>
<BR>
<BR>
<P><FONT SIZE=2>> From: Fab Tillier [<A HREF="mailto:ftillier@silverstorm.com">mailto:ftillier@silverstorm.com</A>] </FONT>
<BR><FONT SIZE=2>> Sent: Wednesday, May 25, 2005 9:00 PM</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Folks,</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> The IPoIB driver is an NDIS miniport driver that reports </FONT>
<BR><FONT SIZE=2>> itself as an 802.3 Ethernet device. NDIS doesn't support MAC </FONT>
<BR><FONT SIZE=2>> addresses larger than 6 bytes.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Currently, the driver has knowledge of how to convert </FONT>
<BR><FONT SIZE=2>> InfiniCon/SilverStorm port GUIDs into valid, globally unique </FONT>
<BR><FONT SIZE=2>> Ethernet MAC addresses. Is this the case for any of the </FONT>
<BR><FONT SIZE=2>> other IB vendors? If so, what is the algorithm?</FONT>
</P>
<P><FONT SIZE=2>Linux 2.4 had the exact same problem. You can take a look </FONT>
<BR><FONT SIZE=2>at the gen1 code to see what of solutions people had. One </FONT>
<BR><FONT SIZE=2>example: gen1/trunk/src/linux-kernel/infiniband/ulp/ipoib/ipoib_arp.c</FONT>
<BR><FONT SIZE=2>is some hashing on the {GID,QPN} into 48 bits. I don't believe it has any</FONT>
<BR><FONT SIZE=2>assumption on how GUID is constructed / allocated by the</FONT>
<BR><FONT SIZE=2>vendor.</FONT>
</P>
<P><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> I'd like to hear suggestions for how to handle all vendors </FONT>
<BR><FONT SIZE=2>> with this driver, as well as how to properly handle GIDs </FONT>
<BR><FONT SIZE=2>> where the lower 64-bits aren't the port GUID.</FONT>
</P>
<P><FONT SIZE=2>Can be one of:</FONT>
<BR><FONT SIZE=2>1) generic hash on GID - all nodes will show the same MAC</FONT>
<BR><FONT SIZE=2> entries on ARP. You're stuck if you have hash collision</FONT>
<BR><FONT SIZE=2>2) create locally faked MAC in each node - not all nodes will</FONT>
<BR><FONT SIZE=2> show the same MAC (arp -a). You can solve it by writing</FONT>
<BR><FONT SIZE=2> a new utility, e.g. ipoib_arp, which will dump the full 20B</FONT>
<BR><FONT SIZE=2> MAC addresses by using some private interface with the NDIS </FONT>
<BR><FONT SIZE=2> miniport</FONT>
<BR><FONT SIZE=2>3) assume some GUID assignment algorithm - can be problematic</FONT>
<BR><FONT SIZE=2> looking into the future. Might not fit all cards, all vendors, etc.</FONT>
<BR><FONT SIZE=2> I'm not sure that all the vendors can commit on how they</FONT>
<BR><FONT SIZE=2> assign GUIDs today, because they may want the ability</FONT>
<BR><FONT SIZE=2> to change the GUID assignment method without rewriting </FONT>
<BR><FONT SIZE=2> the miniport.</FONT>
<BR><FONT SIZE=2>4) Use the 48 bits to contain the path information. A non IB routable</FONT>
<BR><FONT SIZE=2> approach will use the bits as follows: 16 LID, 4 SL, 24 QPN,</FONT>
<BR><FONT SIZE=2> 2 static rate (might be not enough to hold all static rate options)</FONT>
<BR><FONT SIZE=2> 2 reserved (local/global, unicast/mcast) - total 48 bits.</FONT>
<BR><FONT SIZE=2> </FONT>
<BR><FONT SIZE=2>I don't have any strong opinion about which one to choose. </FONT>
</P>
<P><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> One thing that needs to happen is for local instances of the </FONT>
<BR><FONT SIZE=2>> driver to have the same MAC address across disable/enable </FONT>
<BR><FONT SIZE=2>> device state transitions.</FONT>
</P>
<P><FONT SIZE=2>Can you better explain ? MAC address is {GID,QPN}. If you disable/enable</FONT>
<BR><FONT SIZE=2>the HCA you might end up getting a new QPN which implies that your MAC</FONT>
<BR><FONT SIZE=2>address has changed. This is how IPoIB supposed to work.</FONT>
<BR><FONT SIZE=2>If you want to preserve the MAC, you'd need to maintain the same QP which </FONT>
<BR><FONT SIZE=2>is something beyond the standard IB verbs. By tweaking the driver , you can</FONT>
<BR><FONT SIZE=2>reserve some QP numbers for IPoIB. But is it really required ?</FONT>
</P>
<BR>
<P><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Thanks,</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> - Fab</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> _______________________________________________</FONT>
<BR><FONT SIZE=2>> openib-windows mailing list</FONT>
<BR><FONT SIZE=2>> openib-windows@openib.org </FONT>
<BR><FONT SIZE=2>> <A HREF="http://openib.org/mailman/listinfo/openib-windows" TARGET="_blank">http://openib.org/mailman/listinfo/openib-windows</A></FONT>
<BR><FONT SIZE=2>> </FONT>
</P>
</BODY>
</HTML>