[openib-general] [PATCH] kDAPL: cleanup dat/ a bit more

[Grant Grundler]

On Mon, Jun 06, 2005 at 12:15:53PM -0700, Caitlin Bestler wrote:
> As your reply indicates below, the OS does *not*
> know *when* the device is active.

The driver doesn't have to and in fact does not for other technologies
either. The device driver knows (or can know) who registered
the region since that's done in the process context or on behalf
of a kernel user.

> The memory
> is registered as being device accessible for an
> indefinite period of time (must likely until the 
> machine reboots).

Understood. Things leading up to and including Hotplug are generally
the events that would cause us to shudown a card.

> > extreme prejudice if the resources it's sitting on
> > aren't free yet. I don't expect Hotplug support
> > to be simple to implement. But it's definitely possible.
> 
> Killing the task will not promptly cause RDMA activity
> to cease.

Not directly.
It will cause files to get closed, memory to get free'dm and someplace
along the way DMA regions to get unregistered.  In other words, killing
a consumer of a resource should result in the recovery of that resource.
Maybe the libibverbs or libmthca can register a signal handler
to trap such events and clean up properly.


> The memory region must be deallocated or
> the RDMA device handle must be released. So you're
> back to how the kernel verbs are told of this.

I don't know which mechanism can be used to communicate
and handle RDMA region recovery.

> In any event, what you describe would only work for
> shutdown. It would not work to migrate pages. The
> same interface can be used for both purposes.

Ok. I'm only familiar with how to shutdown IO cards.
If someone wants to (and can) implement something more sophisticated
that's great.

grant