[ofa-general] Re: Converting the Linux.au client/server example to a single post_send...

[Roland Dreier]

 > I am attempting to convert your client/server RDMA example code to use a
 > single ibv_post_send() instead of the two calls the client code
 > currently uses.  I'm using the example code from your blog:
 > 
 > http://digitalvampire.org/blog/
 > 
 > What do I have to change?   

I guess you need to create two send work request structs, link the
second one into the first one's next member, and call ibv_post_send()
with the first struct.

 > Separate, but related, I do not understand why the server code does a
 > ibv_post_recv() with the sge.addr set to (buf + sizeof(uint32_t).  This
 > is apparently setting the address to the second word of the buffer.
 > Shouldn't this be the first word?  

No, the client does an RDMA write to write the first value into the
first word, and a send to pass the second value.  So the server will get
the first value in the beginning of the buffer, and it wants to receive
the other value in the second value.

 > Perhaps I misunderstand, but it appears that you are telling the server
 > to start the receive into the second word.  (which implies the data
 > transfer would be out of bounds of buf, clearly not the case as the
 > original example works..  ???)

No, you are right -- the server wants to receive the data into the
second word.  This works fine because the server does

	buf = calloc(2, sizeof (uint32_t));

	mr = ibv_reg_mr(pd, buf, 2 * sizeof (uint32_t),

so the buffer and memory region are both two words long.

 - R.