[ofa-general] RDMA n00b: Remote Memory Access and Connection Setup Help

[Dotan Barak]

Adrien Guillon wrote:
> Hello List,
>
> I've been reading the Infiniband Architecture specification, and I'm 
> ready to write my first RDMA-enabled C++ library.  I have looked at 
> some sample code provided by librdmacm, and I'm having a problem going 
> from the big picture of what I want to do to the finer details.
>
> I want to remotely access memory on various compute nodes, and the 
> Infiniband spec does everything I need.  However upon looking at 
> librdmacm, and the Infiniband verbs I don't see how I actually write 
> to remote memory locations.  Also I don't completely understand why I 
> am responsible for creating QPs in userspace, if everything is 
> supposed to be handled by hardware... although I suppose I still have 
> to allocate memory for the hardware to actually use.
If one wishes to write to remote address:
1) the remote QP access right should support incoming RDMA Write
2) the remote side MR should support Remote Write
3) locally, there is a need to post Send Request with RDMA Write 
operation (and then poll the completion, if such was created)
>
> I'm looking for some help from this list with a high-level overview of 
> what I have to do to establish a connection between two nodes, and how 
> I would write an object to remote memory, or read an object from 
> remote memory.  My current understanding is that I would use librdmacm 
> to create a connection between the nodes that I want to communicate, 
> and then use the inifinband verbs themselves to read/write memory, and 
> to establish a memory window.
You are absolutely right.
Please notice that RDMA Write doesn't consume any Receive Request in the 
Receiver side.

I believe that basin the ucma examples with minor changes (that i wrote 
above) can be written easily.

>
> I'm developing on Linux.
Which is a great OS
:)
>
> Thanks a lot!
You are welcome
Dotan
>
> AJ
> ------------------------------------------------------------------------
>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general