[openib-general] fmr question

[Caitlin Bestler]

openib-general-bounces at openib.org wrote:
>     Steve> So this implies that there is really only one mapping
>     Steve> outstanding at any point in time (less the cache issue).
>     Steve> Right?  So why is there a map count as an fmr attribute?
>     Steve> Its seems like just an arbitrary limit put on how many
>     Steve> times you can map an fmr before unmaping.  -and- once you
>     Steve> unmap you can start mapping again up to the map count... ??
> 
> Part of the L_Key and R_Key are twiddled each time the FMR is
> remapped.  Because of the possibility of stale data hanging
> around the cache, we don't want to reuse an old key without
> flushing any translation caches.  So when we've used all the
> possibilities for the changeable part of the key, we need to
> make sure the FMR is unmapped (and the cache flushed) before
> remapping it. 
> 
> The whole FMR scheme was driven by Mellanox hardware, so if
> there's a way we can change things to make it more generic,
> I'm all for it.
> 

Both the RDMAC and InfiniBand 1.2 verbs define FMR binding
and invalidation through work requests on privileged QPs.
Invalidation of a specific bind is reported as a Work Completion.
By definition, any translation caches MUST be cleared by the
time that completion is delivered to the Consumer.

Adopting those semantics, and making them available via a
Work Request for devices supporting that capability, would
eliminate the problem.

In my opinion undefined flushing of old translation caches
is a security vulnerability that makes the current FMR definition
unusable in any network that was not 100% physically secured.
And even then it is a weakness asking for a bug to hit it.