[openib-general] Re: [RFC] userspace CM/verbs QP

[Sean Hefty]

Libor Michalek wrote:
>> 2. Change the CM API so that it just takes the QPN, QP type, SRQ
>>    status and device directly rather than reading it out of the QP.
>>    This lets the userspace CM just get the info from userspace
>>    without needing to look at the QP at all.  Of course it does raise
>>    the issue of how userspace should specify the device.
> 
>   As you say, the second solution does not resolve the issue about which
> you are worried in the first solution. The device issue I think still
> creates a dependancy between the kernel components of uverbs and ucm,
> it's just moved down the chain, since the only thing that has the user
> to kernel mapping of the device handle is uverbs. Unless that code is
> duplicated, and then it becomes a software maintenance dependency...

The CM needs the device in order to locate which port to send out the 
connection REQ on.  We could let the CM locate the device in the kernel 
based on the user's path record.

This goes back a little to the discussion of having a cm_path field in 
the REQ parameter that the CM can use when sending the REQ.  On the 
receiving side of the REQ, the CM knows the device based on which port 
the REQ came in on.

In order to make this work, there needs to be a call similar to:

ib_find_cached_device_gid(gid, &device, &port_num, &index);

The CM already needs something like this call in order to perform SIDR. 
  (See cm_find_device() in cm.c.)  Support for this call requires some 
changes/exposure to the known device list.

So, I think that we may be able to change the kernel CM API to take the 
  necessary fields, rather than the QP pointer.  I can make doing these 
changes a priority over RMPP if we decide to go this route.

- Sean