[ofiwg] fork support and MR cache

Hefty, Sean sean.hefty at intel.com
Thu Oct 8 12:28:50 PDT 2020

> > I think a reasonable hack could be to implement the behavior from my
> > first email.  That would involve bypassing the libibverbs calls and
> > going directly to the verbs providers for reg/dereg mr.
> If you don't want the MADV then just don't call ibv_fork_init

The issue is that apps (OpenMPI) call ibv_fork_init directly.  The hack Brian is referring to is trying to detect if that has occurred, so the cache can be disabled.

> Please don't recode the kernel interface outside rdma-core, that stuff
> is hard to do right.

I'm was referring to calling:


instead of indirectly through ibv_reg_mr().

That still goes through the rdma-core to reach the kernel.  In theory, we can use this to speed up registrations and enable a cache even if fork support is needed.

I'm actually fine disabling the cache if fork is enabled.  But how can that be detected?
- Sean

More information about the ofiwg mailing list