[openib-general] Re: [PATCH] CMA and iWARP

[Caitlin Bestler]

Steve Wise wrote:
>> Because an iWARP device can encompass multiple Ethernet ports with
>> the same mac address.
> 
> At the same time?  How does a switch handle this?
> 
> I'm not clear on your issue here and why my mac based
> approach breaks it.
> 
> Lets start with a concrete example:
> 
> Lets say I have an rnic device.  It has 2 ports.  It has 1
> mac address per port.  Assume the driver for this device installs 2
> netdevs, one for each port, and there's one ipaddr on each netdev.
> 
> Now, this rnic device will register with the openib core as
> _one_ openib device with 2 ports.
> 
> When the CMA attempts to resolve a route and/or bind to an
> interface, it first consults the routing table.  Lets assume
> it finds one of the netdev devices for this rnic.  It then
> needs to find the associated openib device.  What is wrong
> with using the mac address found in the netdev device and
> finding which openib device has that mac address?  Why does that
> break things? 
> 
> I'm still fuzzy on this issue (and most things ;)
> 
> Given my example above, tell me what delta to that example
> exposes the design flow and helps me understand your issue.
> 
> Sorry if i'm being dumb... :-\
> 

I have two ports, two MAC addresses and two IP addresses.

The cable for the 2nd port breaks, so I move the IP
address to the other MAC address.

If someone does their queries at this point all connections
will appear to be going through the same "port".

When the cable is restored, the IP address will be
moved back to its original MAC address.

Because this process occurs *below* the TCP layer, it is
not visible to the RDMA layer. Therefore the RDMA layer
can only identify the IP address. The RDMA layer does
not need, and should not have knowledge of the L2
address on an IP network.