[openib-general] Re: [PATCH] CMA and iWARP

[Caitlin Bestler]

Steve Wise wrote:
> <stuff deleted>
> 
>>>> By definition iWARP is cleanly layered over IP. Therefore an iWARP
>>>> port is not a physical port but a logical one.
>>>> 
>>>> Management of physical ports is something that must be done
>>>> independently of RDMA software.
>>>> 
>>>> For example, if two physical Ethernet ports are teamed this is NOT
>>>> visible to the RDMA layer. 
>>>> 
>>>> This is a major example of the need to let each transport express
>>>> itself naturally, and finding the common ground that is meaningful
>>>> to applications, rather than forcing one to emulate the other.
>>>> 
>>> 
>>> I'd like us to focus on phase I of iwarp.
>>> 
>>> For phase I, the CMA tries to find an appropriate openib device,
>>> given the dev_addr from the associated netdev that was found during
>>> a routing lookup.  For IB, the dev_addr is matched against gids.
>>> For iwarp, the dev_addr is matched against the mac addr of the
>>> openib dev. 
>>> 
>> 
>> Stop right there.
>> 
>> Once you have the associated netdev you should have 0 or 1
>> associated iWARP RNICs. 
>> 
>> If you go any deeper you risk breaking existing solutions for IP
>> Aliasing, Ethernet teaming, etc.
>> 
> 
> I agree.  However, the question is how to find the associated
> openib device once you determine which netdev device you are
> using for the next hop.  In the existing IB CMA code, this is
> done by a linear search through the ib devices and finding a
> device that has the gid associated with the IPoIB netdev
> device.  I'm proposing we do exactly the same thing, except
> we compare mac addresses.  And we map mac addresses into either gids
> or guids. 
> 
> This is a loosely coupled association between netdevs and open_ib
> devs. However, I think everyone agrees that a netdev maps to only one
> ib device. 
> 
> A tightly coupled design can be considered, but that requires
> more hits into the core netdev code.  For instance, we could
> add a method to the netdev struct to return the openib device
> and thus let the netdev driver tell the CMA exactly which openib
> device it maps to. 
> 
> Steve.

A direct link from the net device to 0 or 1 rdma device would
be better, but if doing it by search then the "mac address"
for iWARP should be the IP address -- not the Ethernet address.