[ofiwg] fork support and MR cache

[Hefty, Sean]

> On Thu, Oct 08, 2020 at 05:45:31PM +0000, Hefty, Sean wrote:
> > > > There have been extensive discussions on github around the MR cache,
> > > > deadlocks, libibverbs madvise tracking, and fork.  The current
> > > > direction is to only enable the MR cache when fork is disabled.
> > > > This was done to work-around internal libibverbs tracking.  But I
> > > > suspect that bypassing that tracking (which is possible) can still
> > > > lead to issues when registrations are made through the MR cache.
> > >
> > > MADV_DONTFORK will be obsolete starting in kernel v5.9
> > >
> > > If you can test and confirm that everything works without it then we
> > > can detect and disable ibv_fork_init on new kernels.
> >
> > Interesting.  What will the behavior be for registered regions when fork is called?
> 
> They are copied into the fork.
> 
> > My concern is that the registrations are made and maintained without
> > the application being aware.  Will cached registrations need to be
> > released when fork is invoked, or is there some other mechanism
> > coming into play now?
> 
> MRs continue to reliably point to memory owned in the parent
> process.
> 
> The child process will be unable to use any MRs or verbs objects, just
> like today.

Thanks - I think this means that fork becomes a non-issue.

- Sean