[ofiwg] fork support and MR cache

[Hefty, Sean]

> > There have been extensive discussions on github around the MR cache,
> > deadlocks, libibverbs madvise tracking, and fork.  The current
> > direction is to only enable the MR cache when fork is disabled.
> > This was done to work-around internal libibverbs tracking.  But I
> > suspect that bypassing that tracking (which is possible) can still
> > lead to issues when registrations are made through the MR cache.
> 
> MADV_DONTFORK will be obsolete starting in kernel v5.9
> 
> If you can test and confirm that everything works without it then we
> can detect and disable ibv_fork_init on new kernels.

Interesting.  What will the behavior be for registered regions when fork is called?

My concern is that the registrations are made and maintained without the application being aware.  Will cached registrations need to be released when fork is invoked, or is there some other mechanism coming into play now?

- Sean