[openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux

[Woodruff, Robert J]

Venkat wrote >   

>It is just unfortunate to make such an IB-centric statement. 
>I will ask the same question back to you - why didn't you consider 
>RNICs when the IB APIs were designed a year ago? 

I did.
I expected that RNICs could plug into the DAPL layer,
as this is what it was designed for. I did not see the need
for an RDMA independent verbs layer, as this is what DAPL is. 
DAPL has already been proven to support multiple RDMA capable devices,
InfiniBand, RNICS (Ammasso), Myrinet, and some others. 

I still think that having a verbs layer for IB and a separate one 
for RNICs is a viable option. At some level, they are different.
We are not talking about a huge amount of code here, only a couple
of thousand lines of code for the core verbs. The rest of the IB
mid-layer,
the CM, SA query, MAD services, are so IB specific, they do not apply
to iWarp and iWarp would need to have their own version of say a CM. 
Thus, I think that the ib_core.ko, ib_cm.ko, ib_sa.ko, ib_mad.ko
layer is the device class specific layer for InfiniBand. An RDMA device 
independent layer needs to be developed above that. 
It is also clear that from a kernel standpoint,
kDAPL was not acceptable since it did not meet the Linux coding
standards. Thus, a major overhaul is already underway to convert
it into the RDMA device independent layer acceptable to Linux. 

>Again because we have developed something a year ago doesn't mean that
>we will have to live with that interface forever. Of course, 
>the question is when and why should this change? As it was discussed 
>before, we would like to see the common PI developed without
>impacting the IB development.

This is true and it has already been stated on this list that we are
open to changes (and the best way to effect change is to 
develop code) that allows RNICs to take advantage of common ULPs. 

I personally would suggest working with the kDAPL people
to evolve that code into something that will work for you, since it 
is under major overhaul already. 

Roland has also stated that he is 
open to modification to the ib_core.ko verbs to support RNICS, so
if that is the approach you want to take and it does not slow down the
InfiniBand development, then, by all means, submit patches to modify
ib_core. Sean expressed some concern that changing the ib_core to also
support 
RNICs would increase complexity and double the testing effort as for 
every change, it would need to be re-tested on both IB and RNICs. 

woody