[ofiwg] [libfabric-users] Shared Rx and Connected

[Doug Oucharek]

For the immediate future, I think I can work around not having readfrom by exchanging some local 64-bit context information as the parameters for connect and accept.  Then, change fi_send calls to fi_senddata and provide the remote’s context as the data.  This will provide an easy way to map an incoming message to a local context.  

Longer term, it seems we could really use another fi_cq_readfrom() specific to connected endpoints.  Rather than return an fi_addr_t, it should return a fid_ep pointer.  It would be great to extend fi_cq_readfrom() with an extra parameter, but that would break existing code so perhaps creating a new call would be best.  Something like:

ssize_t fi_cq_readfromconn(struct fid_cq *cq, void *buf, size_t count,
    struct fid_eq **src_ep);
The local context could be retrieved from the context of the src_ep’s.  

Did you want me to create an issue on this to track it?

Doug


> On Sep 9, 2019, at 11:00 AM, Hefty, Sean <sean.hefty at intel.com> wrote:
> 
>> It seems to me that using a Shared Rx context and 1 CQ with connected endpoints is very
>> much like running connectionless: when a message is received, I cannot see an obvious
>> way to determine the source of the message.  The CQ entry does not have the endpoint in
>> it and I cannot get AVs to work with the TCP or verbs provider (thus I cannot use
>> fi_cq_readfrom).
> 
> I agree - this is an issue.
> 
> 
>> Is there an easy way to determine the source address/endpoint of a received message in
>> this scenario?  Would it make sense to include the ep fid in the CQ entry to help?
> 
> Some sort of extension is needed here.  I'm not sure what the best answer is.  IMO, the ideal solution would make use of fi_cq_readfrom(), so that the source data is reported the same way regardless of EP type.  I just don't see a clean way to do this off the top of my head.
> 
> - Sean

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofiwg/attachments/20190909/1b3e985d/attachment.html>