[openib-general] Question about pinning memory

[Pete Wyckoff]

glebn at voltaire.com wrote on Mon, 25 Jul 2005 18:58 +0300:
> On Mon, Jul 25, 2005 at 11:37:29AM -0400, Pete Wyckoff wrote:
> > I like your signal idea but am not fond of the signal mechanism,
> > especially as we (the library) can't trust the user not to break it.
> > Signaling a separate thread has the same race problem.  I'd prefer
> > to export a counter from the kernel that the user application can
> > see.  The kernel would atomically increment it when there is an
> > unread event, and the app can check the variable to decide if the
> > read() is necessary.
>
> I also don't like signal much but... 
> How userspace will read kernel counter without system call? Or do you
> mean kernel will increment counter resided in userspace?

I'm using copy_to_user() to shadow the number of available VM events
in the queue to an int32 in the user application.  Had considered
mapping a shared page, but that seems like overkill for this tiny
bit of information.  Copy_to_user() is fairly fast and is only
invoked on (hopefully rare) VM activity.  Fortunately VM events
appear only to happen in the context of the user app so this is
possible.  And it drops the fastpath overhead before each MPI_Send()
to around 6 ns on a 2.4 GHz P4 when everything is in cache.

		-- Pete