[openib-general] RFC userspace CMA

[Sean Hefty]

Tom Tucker wrote:
> FYI, I've started writing the iw_cm that sits below the rdma_cm. Here's
> the general picture I have in mind.
> 
> 	+---------+
> 	| RDMA CM |
> 	+-+-----+-+
> 	  |     |
>      +----+     +----+
>      |               |
> +---------+     +----+----+
> | IB CM   |     | IW CM   |
> +----+----+     +----+----+
>      |               |
>  ____+_____      ____+_____
> +---------+|    +---------+|
> | IB devs ||    | IW devs ||
> +---------+     +---------+

This is what I was envisioning as well.

> I am also migrating the current iw_cm.h file to match the interfaces in
> the rdma_cm more closely. 

Note that there are still some changes occurring to the rdma_cm to support 
userspace.  I'm concerned about how well these changes map to iWarp, since the 
changes expose the three-way CM handshake used by IB.

> In general, the IW CM methods look very much like sockets connect,
> listen, and accept. There is an iw_cm_id like the ib_cm_id that
> encapsulates the 5-tuple, a callback for IW CM events and a "provider
> handle" that represents the adapter "connection cookie". The iw_cm_id is
> passed to connect, accept, etc...

Something that didn't make sense for the kernel rdma_cm running over IB was 
adding a backlog parameter to the listen request.  (The IB CM is callback 
driven, so there's not really a backlog.)  I will probably add this to the 
userspace API.  Does iWarp need a backlog parameter in the kernel?

> depending on the model. This means that calls like listen with a local
> port wildcard can't return until the "listen_reply" comes back from the
> adapter.

I didn't quite follow this.  Right now, the rdma_cm only tries to support 
wildcard IP addresses.  Are you wanting to support listening on any port as 
well?  What is a listen_reply?

- Sean