[Openib-windows] Using of fast mutexes in WinIb

[Fab Tillier]

> > One of the main reasons I am pushing for things to work at
> > DISPATCH_LEVEL is to allow kernel clients that get called by
> > a port driver (like StorPort, for
> > example) to make verb calls from the context of their entry
> > points, rather than having to find a way to get into a thread
> > context capable of talking to the HCA.
> 
> That's an ugly scenerio I hadn't thought about. I did think about
> needing to call fabric functions from GetScatterGatherList's
> ExecutionRoute (which the docs say will be at DISPATCH_LEVEL).

In the current driver model you'll have to find a way to get that Scatter Gather
List from the ExecutionRoutine into a passive context if you want to use it to
register memory.  Not a very user friendly scenario.

> > This will push the thread context switch into the HCA if
> > required, rather than duplicating this functionality in all clients.
> 
> That makes it seem easier for clients, but if the HCA is basically just
> passing things to a worker thread at PASSIVE_LEVEL, I'm not sure hiding
> it is better. Is the core issue that some of the IB API's must block to
> perform their function?

The current Mellanox hardware processes commands asynchronously.  The HCA driver
however blocks the calling thread and the DPC that processes the command
completion signals the thread to wake up.  It's purely a SW issue, not a HW one.
In the short term, the thread switch would be moved into the HCA driver to allow
leveraging the existing code.  In the longer term, I expect the HCA driver will
evolve to handle the commands asynchronously (it already does internally), and
these thread context switches will be eliminated all together.

- Fab