[Openib-windows] IPoIB GUID to MAC conversion

[Fab Tillier]

> From: Dror Goldenberg [mailto:gdror at mellanox.co.il]
> Sent: Thursday, May 26, 2005 12:23 AM
> 
> > From: Fab Tillier [mailto:ftillier at silverstorm.com]
> >
> > > Can be one of:
> > > 1) generic hash on GID - all nodes will show the same MAC
> > >       entries on ARP. You're stuck if you have hash collision
> >
> > To add to the collision woes, we probably also want to report
> > valid MAC addresses (globally unique or locally administered
> > - as controlled by the corresponding bit in the 48-bit MAC).
> > If we hash, then the MAC is locally unique, and must have the
> > second to last bit of the first byte set.  It should also not
> > have the least significant bit of the first byte set to
> > indicate that it isn't a multicast MAC address.  That is, the
> > lower two bits of the first byte need to be binary 10.  I
> > don't know what the Windows networking code does with the MAC
> > addresses, and don't know if it's safe to just return full
> > 48-bit hashed values.  I'd prefer to hash to 46 bits.
>
> Right. The two first bits of the MAC address are reserved to
> unicast/mcast and localy/globally administered. We only have
> 46 bits left for tricks. This applies to all 4 approaches.

Correct.  Approach 3, because it creates valid globally unique MAC addresses
honors these bits.

> > > 2) create locally faked MAC in each node - not all nodes will
> > >       show the same MAC (arp -a). You can solve it by writing
> > >       a new utility, e.g. ipoib_arp, which will dump the full 20B
> > >       MAC addresses by using some private interface with the NDIS
> > >       miniport
> >
> > This is similar to the hash as far as reserving bits and what
> > not.  It trades collision for inconsistency.  I think
> > collision is probably better than inconsistency.
>
> Let's think about what collision means for the network administrator. It
> means that he effectively has two cards with the same MAC and therefore
> has to replace one of them or flash it with a newly assigned GUID. If this
> is acceptable, then collision is not a problem.
> You can always argue about the likelihood of this to happen.
> Inconsistency can be solved by having a new utility, new "arp" command,
> that will print the full 20B MAC address, and will make all nodes
> consistent.

I don't know if anyone will care about inconsistency.

> > > 3) assume some GUID assignment algorithm - can be problematic
> > >       looking into the future. Might not fit all cards, all
> > vendors, etc.
> > >       I'm not sure that all the vendors can commit on how they
> > >       assign GUIDs today, because they may want the ability
> > >       to change the GUID assignment method without rewriting
> > >       the miniport.
> >
> > I don't think this should be the only option - if a vendor
> > has an algorithm, it should be used since it generates a
> > valid MAC address.  This option would be used in conjunction
> > with one of the others.
>
> You mean that it can be used to optimize the hashing algorithm ?

Yes - the hashing algorithm would set the LAA bit in the MAC.  Any algorithm
that can create proper globally unique MACs would have this bit clear, so
there's no risk of collision between whatever hashing algorithm is chosen
and any vendor specific algorithm.  The code that generates the Ethernet MAC
would first see if it can generate a globally unique MAC, and if not would
apply the hashing algorithm and generate a LAA MAC.

Note that the InfiniCon/SilverStorm algorithm only uses the port GUID to
generate the MAC.  This is ok even with partitions, since the bus driver
will report a different instance of a NIC for each partition.  Having the
same MAC reported for instances of IPoIB in each partition is OK since each
partition will be a distinct subnet. 

> > > 4) Use the 48 bits to contain the path information. A non
> > IB routable
> > >      approach will use the bits as follows: 16 LID, 4 SL, 24 QPN,
> > >      2 static rate (might be not enough to hold all static
> > rate options)
> > >      2 reserved (local/global, unicast/mcast) - total 48 bits.
> >
> > This is interesting, as it would allow creating static ARP
> > entries.  Does it break the ability of IPoIB routers to route
> > traffic from IB to Ethernet?
>
> No. This approach is IP routable (IPoIB to IPoEther). It is not
> IB routable because we won't be able to add a GID when sending
> packets.

Actually, does it prevent IB global routing?  These generated MAC addresses
are only used to interface with the OS.  On the wire, the MAC is always the
20-byte IPoIB MAC.

- Fab