[libfabric-users] Connection-based vs Connection-less

[Hefty, Sean]

> I have somesimple  newbie questions about libfabrics.  In the context of traditional
> networking, I think I understand connection-based vs connection-less communication
> fairly well.  Connection-based implies that the route between A and B is configured
> before any data communication occurs, and that route stays persists through multiple
> messages.  Connection-less implies that each message finds it's own route through the
> network at the time the message is sent, and that route may change from message to
> message.
> 
> My first question is whether the above description is true for libfabrics connections
> as well?  Are there other key distinctions between connection-based vs connection-less
> communication?

Connection oriented means that the peer endpoints have been configured to communicate with only each other prior to data being transfers.  There is no requirement that the data between the peer always follow the same route though the network.  Libfabric defines additional data and message ordering semantics that permit using multiple routes.

Connectionless allows a single endpoint to communicate with multiple peers.  Every transfer can specify a different peer, which leads us to...

> My second question is about the address vector (AV).  In libfabrics, the AV is required
> for connection-less, but is not used at all for connection-based.  Is that correct?
> What exactly does the address vector do?  What is it's function?

Conceptually, you can think of an AV as an array of peer addresses.  For connectionless transfers, you need to specify the peer's address as part of the request.  You do this by providing the 'index' of the peer's address into the AV array.

Note that an AV doesn't have to be implemented as an array, but it often is.

The *purpose* behind defining an AV is that it allows the API to hide network specific details about how addressing works.


> Finally, in the fabtests examples, I looked at rdm_rma_simple.c (along with shared.c)
> and pruned it to get rid of the unnecessary code.  I still can't figure out if
> rdm_rma_simple.c is connection-less or connection-based.  I think it's connection-less,
> but there is a lot of code to setup the connection beforehand.  Is it indeed
> connection-less?

The 'rdm' in the name means that it uses reliable-datagram (i.e. RDM) endpoints.  That is a reliable, connectionless endpoint type.

> Assuming rdm_rma_simple.c is connection-less, which of the fabtests would be a simple
> connection-based example?  I want to start out with the simplest fabtests code that is
> connection-based.

fi_msg is a connection oriented example.  Note that because fabtests shares a lot of code between tests, many support both connection-oriented and connectionless transfers.

You may also want to look at the libfabric util tests, pingpong.  That supports both.

- Sean