[openib-general] RFC userspace CMA

[Tom Tucker]

Sean:

FYI, I've started writing the iw_cm that sits below the rdma_cm. Here's
the general picture I have in mind.

	+---------+
	| RDMA CM |
	+-+-----+-+
	  |     |
     +----+     +----+
     |               |
+---------+     +----+----+
| IB CM   |     | IW CM   |
+----+----+     +----+----+
     |               |
 ____+_____      ____+_____
+---------+|    +---------+|
| IB devs ||    | IW devs ||
+---------+     +---------+

The purpose of the IW CM is to abstract the two different connection
models used by the iWARP side: offloaded and host integrated, and to act
as a shim between device specific connection data structures and the
rdma_cm data structures.

I am also migrating the current iw_cm.h file to match the interfaces in
the rdma_cm more closely. 

In general, the IW CM methods look very much like sockets connect,
listen, and accept. There is an iw_cm_id like the ib_cm_id that
encapsulates the 5-tuple, a callback for IW CM events and a "provider
handle" that represents the adapter "connection cookie". The iw_cm_id is
passed to connect, accept, etc...

One big difference between the IW_CM and the IB_CM is that the IW_CM
does not implement the connection state machine (three way handshake)
like it does in the IB_CM. This greatly simplifies the code. Another big
difference is that the IW_CM does not implement the service id database
(port number space). This is either in the adapter or native stack
depending on the model. This means that calls like listen with a local
port wildcard can't return until the "listen_reply" comes back from the
adapter.

I welcome all comments on this especially now that it's early and
there's a lot of options and not much code yet.


On Tue, 2005-10-25 at 16:06 -0700, Sean Hefty wrote:
> Sean Hefty wrote:
> > - The kernel CMA will expose a new call, rdma_init_qp_attr() to 
> > initialize QP attributes used to modify the state of the QP.  The call 
> > will be similar to the infiniband CM routine.  Use of this call is 
> > optional.  The CMA will automatically transition QPs created by 
> > rdma_create_qp().
> 
> The changes are more involved than this.  To handle the QP transitions in 
> userspace, the kernel CMA needs to generate another event: CONNECT_RESPONSE.  It 
> will also need an additional API: rdma_establish().  (We can overload 
> rdma_accept() in place of rdma_establish().)  Basically, the 3-way handshake 
> used by IB needs to be exposed.
> 
> Use of either of these can be limited to those users that do not associate a QP 
> with their rdma_cm_id.  Alternatively, the uCMA kernel component can be 
> integrated with the kernel CMA and make use of private interfaces.
> 
> - Sean
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general