[openib-general] OpenIB and OpenRDMA: Convergence on common RDMA APIs and ULPs for Linux

[Michael Krause]

At 09:49 AM 5/26/2005, Sean Hefty wrote:
>Roland Dreier wrote:
>>I believe the way forward is to evolve the existing drivers/infiniband
>>code already in Linux into a drivers/rdma that supports both IB and
>>RNICs.  To be extremely blunt, I believe the RNIC-PI is irrelevant to
>>the Linux kernel -- no IB vendors will support ripping out a working
>>midlayer and starting from scratch, and it doesn't make sense to have
>>two essentially equivalent midlayers in the same kernel.
>
>IMO, any APIs need to evolve out of the implementation.  Trying to fit an 
>implementation under an existing API tends to lead to either a poor 
>implementation, or requires changes to the API anyway.
>
>I think a useful path is for someone to implement an RNIC driver and 
>provide feedback on what changes would be required of the Infiniband/core 
>layer to support it.

There needs to be a balance between establishing a good, flexible 
architecture and how applications or subsystems interact with it.  All of 
this needs to be grounded in implementation experience so there is a 
healthy does of reality.  The problem with the iterative-TTM focused API 
definitions is they rarely produce something that is focused on the "end 
game" for a solid interface.  They create instant legacy baggage that 
people are unwilling to shed as time goes forward.  This legacy inertia is 
what has stifled quite a bit of innovation when it comes to software (some 
hazard that 90% of the software created today is focused primarily on 
legacy investment protection than really new innovation and when you think 
about it for a bit of time, you can see that much of this is re-inventing 
the wheel or to band-aid over a problem and packaging it as something 
innovative).

So, the problem becomes one of finding balance.  The people proposing the 
various API are people who have implemented various types of solutions so 
they are coming with both real experience as well as engineering judgement 
of what is required to get the right infrastructure in place for the "end 
game" needs.  From a practical perspective, one needs to implement these 
API and see what is really of value and what should be changed.  The key is 
to avoid creating the legacy inertia that ends up reducing fragmenting the 
interfaces and causes lost productivity, quality problems, etc. So before 
people write off the various API as unnecessary or as poor quality, there 
should be some implementations developed and some constructive debate of 
what features are really of value and what might be deprecated or 
eliminated.  There is no requirement to ever implement all of an API as 
this is where an iterative approach has value.  Implement what is needed to 
demonstrate the desired value and decide then if it is worthwhile.  Just 
please don't discount it or toss it all out because it isn't all 
implemented today or you don't like some aspect.


>>To put a really concrete proposal on the table, I would suggest to
>>start by extending the current ib_client registration structure
>
>Roland's proposal sounds like a reasonable and fair way to 
>begin.  Building an abstraction on top of the existing layers seems 
>secondary to adding support for other RDMA devices.

Getting the right verbs interface is paramount.  Whether one does IT API or 
DAPL on top of that for a given subsystem is secondary.

Ideally, OpenIB and OpenRDMA should be focused on developing the RDMA 
infrastructure - RDMA verbs, connection management, IB-specific ULP, 
etc.  They should not be focused on the upper subsystems or ULP.  Those 
should be done as separate open source projects and they can decide where 
best to intersect with the Open* infrastructure.  This one-size-fits-all 
approach for all subsystems to flow through a given interface is simply 
impractical to execute.  It might happen over time but it cannot be 
force-fit.  Let's focus the two efforts on getting as common of an 
infrastructure as possible.  Some propose RNIC PI; other can propose 
something else if there is desire.  RNIC PI has value in that it has 
focused on the common components between IB and iWARP and left the 
IB-specific and CM outside of its definition to allow an OS to decide how 
best to support or to allow IB to do what is needed for its particular usages.


Mike 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050526/98e96683/attachment.html>