[libfabric-users] multi-receive CQ entry with FI_MULTI_RECV, not the last one read by fi_cq_read()?

[Hefty, Sean]

> > I post a multi-receive buffer via fi_recvmsg(..., FI_MULTI_RECV) and then process it
> by
> > reading CQ entries which refer to the messages landing there. When I see a CQ entry
> > with FI_MULTI_RECV in its flags I re-post that same multi-receive buffer with
> > fi_recvmsg() again. Based on reading the man pages, I've put an assertion in my CQ
> > entry processing that if an entry has FI_MULTI_RECV in its flags, that entry must be
> > the last one my fi_cq_read() read. Essentially, this assertion is to confirm my
> > understanding that nothing can be placed in the multi-receive buffer after the
> provider
> > releases it.
> 
> This should be the case.  What version of the library are you using?
> 
> Are you posting more than 1 multi-recv buffer?  There were issues in older code where
> providers were reposting multi-recv buffers to the end of the receive queue.  So, if
> there were 2 buffers posted, received data would intermingle between them.  (Data
> ordering was unaffected).

In addition, there was also an issue in the multi-recv posting path not handling unexpected messages correctly.  I don't remember which version of libfabric correct the above issues, but they should be fixed in the upstream code.

- Sean