[libfabric-users] multi-receive CQ entry with FI_MULTI_RECV, not the last one read by fi_cq_read()?

[Titus, Greg]

No, there is only the one buffer, which is posted at the beginning of time and then re-posted whenever I read a FI_MULTI_RECV CQ event.  And while buf[2].buf through buf[4].buf are earlier in the buffer than buf[0].buf, they are not anywhere near the beginning of it.

greg
________________________________
From: Hefty, Sean <sean.hefty at intel.com>
Sent: Wednesday, June 24, 2020 4:50 PM
To: Titus, Greg <gregory.titus at hpe.com>
Cc: libfabric-users at lists.openfabrics.org <libfabric-users at lists.openfabrics.org>
Subject: RE: multi-receive CQ entry with FI_MULTI_RECV, not the last one read by fi_cq_read()?

> I called fi_cq_read(cq, buf, 5) and it filled in buf with 5 entries.
> buf[0].flags==FI_MSG|FI_RECV and buf[0].buf was near the end of the multi-receive
> buffer.  buf[1].flags==FI_MULTI_RECV.  buf[2].flags==FI_MSG|FI_RECV and buf[2].buf was
> much earlier in the multi-receive buffer than buf[0].buf.  buf[3] and buf[4] were the
> same as buf[2] but with different .buf pointers though both were also much earlier in
> the multi-receive buffer than buf[0].buf.

Are you sure that the buffer was only posted once?  When is the buffer reposted?

The scenario you're describing is possible if the buffer is posted twice, or if a single buffer were split and posted as two receives.  For example:

If buf[0].buf + buf[0].len is near the end of the buffer...

buf[1] would mark the buffer as done (FI_MULTI_RECV set)

buf[2].buf would then reference the start of the buffer again
buf[3].buf would be near buf[2].buf + buf[2].len
buf[4].buf ~ buf[3].buf + buf[3].len
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20200624/4149e518/attachment.htm>