[ofiwg] shared recvs

[Sur, Sayantan]

The manpage on fi_endpoint says that scalable endpoints have only one transport level address. It seems to me that in your example using passive endpoints, you would end up with two transport addresses. I'm not sure that the scalable endpoint is what you want here. Are you wanting the ability to use FI_RDM with passive endpoint?

I think in general, there might be an issue with sharing the same receive queue (i.e. FI_RDM) when using passive endpoints. It seems to me from the APIs that every time you accept a new "connection" you end up with a new endpoint - like in a connection oriented API. Since every endpoint has its own receive queue, there isn't a good way to share that receive queue.

I'm not sure what the solution is here, other than to wait for Sean :) It could be that the fi_cm APIs need a mechanism to "reuse" an open RDM type endpoint, vs creating a new one each time.

From: ofiwg-bounces at lists.openfabrics.org [mailto:ofiwg-bounces at lists.openfabrics.org] On Behalf Of Reese Faucette (rfaucett)
Sent: Tuesday, October 28, 2014 12:25 PM
To: ofiwg at lists.openfabrics.org
Subject: [ofiwg] shared recvs

If an app wishes to create a passive endpoint, accept connections on it, and then post receives that will receive data coming from any remote connection on that endpoint, how exactly is that accomplished?

My best guess is by using fi_rx_context, we can post a "shared" receive buffer via:

fi_pendpoint(&pep);
fi_bind(pep, cmeq);
fi_listen(pep);

fi_eq_sread(cmeq); // wait for CONNREQ 1
fi_endpoint(&cep1);  // remote-specific EP
fi_accept(cep1);
fi_eq_sread(cmeq); // wait for COMPLETE 1

fi_eq_sread(cmeq); // wait for CONNREQ 2
fi_endpoint(&cep2);  // remote-specific EP
fi_accept(cep2);
fi_eq_sread(cmeq); // wait for COMPLETE 2

fi_rx_context(cep1, 0, &rxep);   // get common EP for RX
fi_recv(rxep, buf, len);

Now, a send from the remote endpoint associated with either cep1 or cep2 will land in buf, yes ?

I'm sure there are cases where support for this mode of operation is desired vs. not, what are the endpoint flags that would control whether this approach will work or not?  I imagine that an endpoint that supports the above would NOT support posting a receive directly to cep1 or cep2, and that endpoint that expect to post received to cep1 only for remote1 and cep2 for remote 2 (e.g. traditional Verbs RC endpoint) would not support this shared mode of operation.

Assuming I am not too far off in the woods here, how is this shared/non-shared approach to receives communicated in the API?
Thanks,
-reese

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofiwg/attachments/20141028/affabe8c/attachment.html>