[openib-general] Re: device add/remove in userspace

[Michael S. Tsirkin]

Quoting r. Sean Hefty <mshefty at ichips.intel.com>:
> Subject: Re: device add/remove in userspace
> 
> Roland Dreier wrote:
> >     Sean> Is there a way for a userspace application to know if a
> >     Sean> device has been added or removed?
> > 
> > We don't really handle this right now.  It could probably be made to
> > work on top of hotplug/udev/hal/something but it seems tricky to me.
> 
> Any idea on how we currently want to handle this?  Can a device remove occur if 
> a usermode app is using the device?

Well, an easy way out would be to pass an event up to the application,
and block device removal until the application closes all resources.
This doesnt seem to be hard to do, given exiting mechanisms of passing
asynchronous events up to userspace.

Ideally, of course, we would free the device without waiting for userspace,
but that may be trickier to accomplish: probably need to modify all
QPs to error state, then close them and close all CQs.
Further, need to unmap the device memory mapped into application's memory.

> The kernel CMA will block until all connection identifiers associated with the 
> removed device are freed.  I export the device removal up to userspace through 
> the uCMA, but that leaves the kernel blocked until the userspace application 
> cleans up.  Doing something more intelligent requires a lot more effort.
> 
> - Sean

I think this is reasonable, short-term.

-- 
MST