[libfabric-users] ibv_post_send() execution time

Dave Goodell (dgoodell) dgoodell at cisco.com
Sat Sep 3 07:17:41 PDT 2016

Hi Cheng,

Your question is about libibverbs but this list is for libfabric. Perhaps someone else on this list knows the appropriate venue for your question.


On Sep 2, 2016, at 11:57 PM, Cheng Wang <wangchenghku at gmail.com<mailto:wangchenghku at gmail.com>> wrote:


I am new to OFI and I am doing some microbenchmarks.

I measured the execution time of ibv_post_send() in both rping(ping-pong) program and ib_write_lat program. Below is my pseudo code:

clock_gettime(); // start
clock_gettime(); // end

Specifically, I measured IBV_WR_RDMA_WRITE operation using the same data size in Reliable Connection (RC) mode.

And the result shows each ibv_post_send() in rping takes around 170~180 nanoseconds to return, but in ib_write_lat program it only takes around 60 nanoseconds to return in average.
I suspected that this is caused by memalign of the sent buffer in ib_write_lat. But after I changed the rdma sent buffer in rping to be also memory aligned, it still takes around 170 nanoseconds.

May I know what caused this difference?

Many thank in advance.
Libfabric-users mailing list
Libfabric-users at lists.openfabrics.org<mailto:Libfabric-users at lists.openfabrics.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20160903/6f0c8fde/attachment.html>

More information about the Libfabric-users mailing list