All, I've added logic to krping to use fast_reg_mrs, local_inv, as well as send-with-inv . You can pull the code from git://git.openfabrics.org/~swise/krping.git There is a README that tries to explain the test case. Enjoy. If you have comments/criticisms, please include a patch. :) Steve.