[openib-general] CMA and device removal

[Sean Hefty]

>I think this would require some modification of our current device
>removal handling.  Currently we issue removal notifications in reverse
>order of client registration -- in other words, the client that
>registered last gets notified first.  This means that we don't notify
>CMA of a device removal until after everyone using CMA has already
>supposedly cleaned up.

The idea with this is that a user of the CMA does not need to register for
device addition/removal, and track devices themselves.  What I have right now is
something similar to this:

rdma_create_id();
rdma_bind_addr(id, optional src addr, dst addr);
rdma_resolve_route(id); /* optional - done by connect if not called */
rdma_connect(id);

and on the passive side:

rdma_create_id();
rdma_bind_addr(id, src addr);
rdma_listen(id);

Bind completes asynchronously.  On the active side, bind associates an id with a
local RDMA device (available through id->device).  On the passive side, bind may
or may not associate the listen id with a device, depending if a source address
was specified, or only a port.  If the device associated with an id is removed,
a device removal event is generated to the user.

- Sean