[ofiwg] fork support and MR cache

[Jason Gunthorpe]

On Thu, Oct 08, 2020 at 05:45:31PM +0000, Hefty, Sean wrote:
> > > There have been extensive discussions on github around the MR cache,
> > > deadlocks, libibverbs madvise tracking, and fork.  The current
> > > direction is to only enable the MR cache when fork is disabled.
> > > This was done to work-around internal libibverbs tracking.  But I
> > > suspect that bypassing that tracking (which is possible) can still
> > > lead to issues when registrations are made through the MR cache.
> > 
> > MADV_DONTFORK will be obsolete starting in kernel v5.9
> > 
> > If you can test and confirm that everything works without it then we
> > can detect and disable ibv_fork_init on new kernels.
> 
> Interesting.  What will the behavior be for registered regions when fork is called?

They are copied into the fork.

> My concern is that the registrations are made and maintained without
> the application being aware.  Will cached registrations need to be
> released when fork is invoked, or is there some other mechanism
> coming into play now?

MRs continue to reliably point to memory owned in the parent
process.

The child process will be unable to use any MRs or verbs objects, just
like today.

Jason