[ofa-general] Re: Demand paging for memory regions

[Steve Wise]

Felix Marti wrote:

> 
> That is correct, not a change we can make for T3. We could, in theory,
> deal with changing mappings though. The change would need to be
> synchronized though: the VM would need to tell us which mapping were
> about to change and the driver would then need to disable DMA to/from
> it, do the change and resume DMA.
> 

Note that for T3, this involves suspending _all_ rdma connections that 
are in the same PD as the MR being remapped.  This is because the driver 
doesn't know who the application advertised the rkey/stag to.  So 
without that knowledge, all connections that _might_ rdma into the MR 
must be suspended.  If the MR was only setup for local access, then the 
driver could track the connections with references to the MR and only 
quiesce those connections.

Point being, it will stop probably all connections that an application 
is using (assuming the application uses a single PD).


Steve.