[openib-general] Re: Re: Question about pinning memory

[Michael S. Tsirkin]

Quoting r. Gleb Natapov <glebn at voltaire.com>:
> > Having said that - do you have some benchmarks to prove that
> > pinning is that expensive? Are you sure that all these internal table
> > lookups aren't even slower?
> > I cant promise, but given a microbenchmark, maybe something could be done to
> > improve the speed of this operation.
>
> In my experience cache hit in MVAPICH improves registration by factor of
> two.

But thats VAPI based, not gen2 based, isnt it?

> Internal table lookup is a search in balanced tree. Hardware memory
> registration is system call + writes on PCI bus + search in balanced
> tree in get_user_pages,

I cant tell without seeing actual benchmarks.
Certainly tricks like overriding malloc/free operations can have
impact on overall application performance.

> how can you improve this?

I have some optimization ideas in mind - clearly even with a very smart
registration cache there may be cache misses, so if a problem exists,
any optimizations shall be beneficial.


-- 
MST