[openib-general] RDMA Generic Connection Management

[Guy German]

Hi,

After receiving feedbacks from people here - I want to see if we can
agree on a generic CM API, so we can start implementing it. 
I will try and summarize the 2 options, the way I understand it. 

If I am missing something or misrepresenting - please don't hesitate to
correct me.

both suggestion include the following verbs (or semantically
equivalent): ib_cma_get_device, ib_cma_create_qp, ib_cma_connect,
ib_cma_disconnect, ib_cma_listen, ib_cma_destroy, ib_cma_accept,
ib_cma_reject, ib_cma_get_src_ip.

a connect flow will be something like:

- ib_cma_get_device (...) /* get device(1) or device+path(2) */
- pd = ib_alloc_pd(...) /* pd allocated in the given device */
- qp = ib_cma_create_qp(...) /* qp returned in init state */
- ib_post_recv(qp, ...);
- ib_cma_connect (qp, dst_addr(1)/path(2), ...);

Now, there are 2 suggestions for the device discovery:
1. get_device returns device and port, according the local routing
tables, synchronously. ib_cma_connect calls the at module for address
resolving (cache handled) before calling the cm_connect.
2. get_device registers an upcall and return in the upcall the data path
to the consumer. In this case caching is done by the consumer.

I prefer option 1, because it makes the consumer code simpler, without
having to handle upcalls for address translations (which are not
asynchronous in iWARP) or hold the transport's data information. Also it
saves the consumer the trouble of caching routes to destinations.

I would like to hear what other people in the list think of it ...

Thanks,
Guy