[Openib-windows] IPoIB GUID to MAC conversion

[Fab Tillier]

> From: Dror Goldenberg [mailto:gdror at mellanox.co.il]
> Sent: Wednesday, May 25, 2005 12:32 PM
> 
> > From: Fab Tillier [mailto:ftillier at silverstorm.com]
> > Sent: Wednesday, May 25, 2005 9:00 PM
> >
> > Currently, the driver has knowledge of how to convert
> > InfiniCon/SilverStorm port GUIDs into valid, globally unique
> > Ethernet MAC addresses.  Is this the case for any of the
> > other IB vendors?  If so, what is the algorithm?
> 
> Linux 2.4 had the exact same problem. You can take a look
> at the gen1 code to see what of solutions people had. One
> example: gen1/trunk/src/linux-kernel/infiniband/ulp/ipoib/ipoib_arp.c
> is some hashing on the {GID,QPN} into 48 bits. I don't believe it has any
> assumption on how GUID is constructed / allocated by the
> vendor.

Right - it just does a hash (option 1 in your list).

> >
> > I'd like to hear suggestions for how to handle all vendors
> > with this driver, as well as how to properly handle GIDs
> > where the lower 64-bits aren't the port GUID.
> 
> Can be one of:
> 1) generic hash on GID - all nodes will show the same MAC
>       entries on ARP. You're stuck if you have hash collision

To add to the collision woes, we probably also want to report valid MAC
addresses (globally unique or locally administered - as controlled by the
corresponding bit in the 48-bit MAC).  If we hash, then the MAC is locally
unique, and must have the second to last bit of the first byte set.  It
should also not have the least significant bit of the first byte set to
indicate that it isn't a multicast MAC address.  That is, the lower two bits
of the first byte need to be binary 10.  I don't know what the Windows
networking code does with the MAC addresses, and don't know if it's safe to
just return full 48-bit hashed values.  I'd prefer to hash to 46 bits.

> 
> 2) create locally faked MAC in each node - not all nodes will
>       show the same MAC (arp -a). You can solve it by writing
>       a new utility, e.g. ipoib_arp, which will dump the full 20B
>       MAC addresses by using some private interface with the NDIS
>       miniport

This is similar to the hash as far as reserving bits and what not.  It
trades collision for inconsistency.  I think collision is probably better
than inconsistency.

> 
> 3) assume some GUID assignment algorithm - can be problematic
>       looking into the future. Might not fit all cards, all vendors, etc.
>       I'm not sure that all the vendors can commit on how they
>       assign GUIDs today, because they may want the ability
>       to change the GUID assignment method without rewriting
>       the miniport.

I don't think this should be the only option - if a vendor has an algorithm,
it should be used since it generates a valid MAC address.  This option would
be used in conjunction with one of the others.

> 
> 4) Use the 48 bits to contain the path information. A non IB routable
>      approach will use the bits as follows: 16 LID, 4 SL, 24 QPN,
>      2 static rate (might be not enough to hold all static rate options)
>      2 reserved (local/global, unicast/mcast) - total 48 bits.

This is interesting, as it would allow creating static ARP entries.  Does it
break the ability of IPoIB routers to route traffic from IB to Ethernet?  I
personally don't give a hoot about IB global routing.  I think it's a dumb
feature and adds needless complexity to the architecture.

> >
> > One thing that needs to happen is for local instances of the
> > driver to have the same MAC address across disable/enable
> > device state transitions.
> 
> Can you better explain ? MAC address is {GID,QPN}. If you disable/enable
> the HCA you might end up getting a new QPN which implies that your MAC
> address has changed. This is how IPoIB supposed to work.
> If you want to preserve the MAC, you'd need to maintain the same QP which
> is something beyond the standard IB verbs. By tweaking the driver , you
> can reserve some QP numbers for IPoIB. But is it really required ?

I meant the Ethernet MAC address reported to the OS.  Method 2 can and
method 4 will generate different MAC addresses when an instance is
re-enabled.  On the wire, any packets (like ARP) that include the MAC
include the 20-byte IPoIB MAC, not the generated 6-byte Ethernet MAC.
Perhaps the network stack doesn't care about a MAC address changing every
time the device is enabled.  I don't know.

- Fab