[libfabric-users] ibv_post_send() execution time

[Dave Goodell (dgoodell)]

Hi Cheng,

Your question is about libibverbs but this list is for libfabric. Perhaps someone else on this list knows the appropriate venue for your question.

-Dave

On Sep 2, 2016, at 11:57 PM, Cheng Wang <wangchenghku at gmail.com<mailto:wangchenghku at gmail.com>> wrote:

Hi,

I am new to OFI and I am doing some microbenchmarks.

I measured the execution time of ibv_post_send() in both rping(ping-pong) program and ib_write_lat program. Below is my pseudo code:

clock_gettime(); // start
ibv_post_send();
clock_gettime(); // end

Specifically, I measured IBV_WR_RDMA_WRITE operation using the same data size in Reliable Connection (RC) mode.

And the result shows each ibv_post_send() in rping takes around 170~180 nanoseconds to return, but in ib_write_lat program it only takes around 60 nanoseconds to return in average.
I suspected that this is caused by memalign of the sent buffer in ib_write_lat. But after I changed the rdma sent buffer in rping to be also memory aligned, it still takes around 170 nanoseconds.

May I know what caused this difference?

Many thank in advance.
_______________________________________________
Libfabric-users mailing list
Libfabric-users at lists.openfabrics.org<mailto:Libfabric-users at lists.openfabrics.org>
http://lists.openfabrics.org/mailman/listinfo/libfabric-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20160903/6f0c8fde/attachment.html>