[openib-general] Re: [PATCH][RFC][0/4] InfiniBand userspace verbsimplementation

[Bodo Eggert]

On Fri, 22 Apr 2005, Fab Tillier wrote:
> > From: Bodo Eggert <harvested.in.lkml at posting.7eggert.dyndns.org>
> > Sent: Friday, April 22, 2005 6:10 AM

> > You can't even set a time limit, the driver may have allocated all DMA
> > memory to queued transfers, and some media needs to get plugged in by
> > the lazy robot. As soon as the robot arrives - boom. (For the same reason,
> > this memory MUST NOT be freed if the application terminates abnormally,
> > e.g. killed by OOM).
> 
> InfiniBand provides support for deregistering memory that might be
> referenced at some future time by an RDMA operation.  The only side effect
> this has is that the QP on both sides of the connection transition to an
> error state.
> 
> Upon abnormal termination, all registrations must be undone and the memory
> unpinned.  This must be synchronized with the hardware so that there are no
> races.

If you know the hardware. If you have userspace drivers, this will be
impossible, and even if you have kernel drivers, you'll need to know 
which of them is responsible for each part of the pinned memory.

This doesn't imply the affected memory to be lost. The same application
that created the pinned memory can reset the hardware (provided nobody
changed the configuration), then reconnect to the shared memory segment
you'll use for that purpose and use or free it.

-- 
To iterate is human; to recurse, divine.