[Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux

[Caitlin Bestler]

On 6/1/05, Christoph Hellwig <hch at lst.de> wrote:
> On Wed, Jun 01, 2005 at 09:36:09AM +0200, Christoph Hellwig wrote:
> > For (1) doing a proper RDMA stack should solve thing, and the discussion
> > how to do it is already ongoing on this list.  Once we have proper RMDA
> > stack that part of KDAPL isn't needed at all anymore.  The second is
> > more interesting and there's indeed the need for some higher level
> > helpers than the API at the level of the current OpenIB code offers. But
> > having a separate layer, with different data structures, provider
> > registration and a totally different API for that is utter nonsense.
> > Instead the higher level helpers should operate on the same
> > datastructures as the RDMA stack, or build new ones ontop of that.
> > In addition some of the abstractions don't make much sense, the event
> > handling has already been mentioned.
> 
> That beeing said, one of the first thing you should get rid of if you
> want to be able to take code from kdapl to the generic rdma code is way
> it deals with handles.  The kdapl code gives up language type safety
> just to add its own bandaided type system below.  Please always pass the
> real struct pointer around, or at least the generic struct type which
> might be embedded into a bigger structure, and get rid of all the magic
> number checking and magic macros that expect special members at the
> beginning of all objects behind these handles.
> 

That is a good suggestion, and is fully compatible with DAT defnitions.
In fact the first version of the reference implementation did use struct
pointers as the DAT handles. The indirection method was added after
the fact to reduce problems with bad pointers in sloppy user-mode
applications, and is specifically something that any DAT implementation
is free to revert back to the struct pointer.

I fully agree that having the indirected handled is a layer of safety that
is not called for in a kernel (or any embedded application for that matter).
DAT Consumers are not supposed to rely upon such safety mechanisms,
and the DAT handle is opaque to them. So changing the representation
of the DAT Handle is *totally* legitimate.