[openib-general] Re: [PATCH] CMA and iWARP

[Steve Wise]

<stuff deleted>

> >> By definition iWARP is cleanly layered over IP. Therefore an iWARP
> >> port is not a physical port but a logical one.
> >> 
> >> Management of physical ports is something that must be done
> >> independently of RDMA software.
> >> 
> >> For example, if two physical Ethernet ports are teamed this is NOT
> >> visible to the RDMA layer. 
> >> 
> >> This is a major example of the need to let each transport express
> >> itself naturally, and finding the common ground that is meaningful to
> >> applications, rather than forcing one to emulate the other.
> >> 
> > 
> > I'd like us to focus on phase I of iwarp.
> > 
> > For phase I, the CMA tries to find an appropriate openib
> > device, given the dev_addr from the associated netdev that
> > was found during a routing lookup.  For IB, the dev_addr is
> > matched against gids.  For iwarp, the dev_addr is matched
> > against the mac addr of the openib dev.
> > 
> 
> Stop right there.
> 
> Once you have the associated netdev you should have 0 or 1
> associated iWARP RNICs.
> 
> If you go any deeper you risk breaking existing solutions
> for IP Aliasing, Ethernet teaming, etc.
> 

I agree.  However, the question is how to find the associated openib
device once you determine which netdev device you are using for the next
hop.  In the existing IB CMA code, this is done by a linear search
through the ib devices and finding a device that has the gid associated
with the IPoIB netdev device.  I'm proposing we do exactly the same
thing, except we compare mac addresses.  And we map mac addresses into
either gids or guids.

This is a loosely coupled association between netdevs and open_ib devs.
However, I think everyone agrees that a netdev maps to only one ib
device.

A tightly coupled design can be considered, but that requires more hits
into the core netdev code.  For instance, we could add a method to the
netdev struct to return the openib device and thus let the netdev driver
tell the CMA exactly which openib device it maps to.  

Steve.