[ofa-general] Memory registration redux

[Roland Dreier]

 > > /*
 > >  * If type field is INVAL, then user_cookie_counter holds the
 > >  * user_cookie for the region being reported; if the HINT flag is set
 > >  * then hint_start/hint_end hold the start and end of the mapping that
 > >  * was invalidated.  (If HINT is not set, then multiple events
 > >  * invalidated parts of the registered range and hint_start/hint_end
 > >  * should be ignored)

 > I don't quite grok this.  Is the intent that HINT will only be set if
 > an *entire* hint_start/hint_end range is invalidated by a single
 > event?  I.e., if only part of the hint_start/hint_end range is
 > invalidated, you'll get the cookie back, but not what part of the
 > range is invalid (because assumedly the entire IBV registration is now
 > invalid anyway)?

Basically, I just keep one hint_start/hint_end.  If multiple events hit
the same registration then I just give up and don't give you a hint.

 > >  * If type is LAST, then the read operation has emptied the list of
 > >  * invalidated regions, and user_cookie_counter holds the value of the
 > >  * kernel's generation counter when the empty list occurred.  The
 > >  * other fields are not filled in for this event.

 > Just to be clear -- we're supposed to keep reading events until we get
 > a LAST event?

Yes, that's probably the sanest use case.

 > 1. Will it increase by 1 each time a page (or set of pages?) is
 > removed from a user process?

As it stands it increases by 1 every time there is an MMU notification,
even if that notification hits multiple registrations.  It wouldn't be
hard to change that to count the number of events generated if that
works better.

 > 2. Does it change if pages are *added* to a user process?  I.e., does
 > the counter indicate *removals* or *changes* to the user process page
 > table?

No, additions don't trigger any MMU notification -- that's inherent in
the design of the MMU notifiers stuff.  The idea is that you have a
"secondary MMU" and MMU notifications are the equivalent of TLB
shootdowns; the secondary MMU is responsible for populating itself on
faults etc.

 > Is the *unm_counter value guaranteed to have been changed by the time
 > munmap() returns?

Yes.

 > Did you pick [2] here simply because you're only expecting an INVAL
 > and a LAST event in this specific example?  I'm assuming that we
 > should normally loop over reading until we get LAST, correct?

Right.

 > What happens if I register multiple regions with the same cookie value?

You get in trouble -- I need to fix things to reject duplicated cookies
actually, because otherwise there's no way to unregister.

 > Is a process responsible for guaranteeing that it umn_unregister()s
 > everything before exiting, or will all pending registrations be
 > cleaned up/unregistered/whatever when a process exits?

The kernel cleans up of course to handle crashes etc.

 - R.