[libfabric-users] multi-receive CQ entry with FI_MULTI_RECV, not the last one read by fi_cq_read()?
sean.hefty at intel.com
Wed Jun 24 12:04:00 PDT 2020
> > I post a multi-receive buffer via fi_recvmsg(..., FI_MULTI_RECV) and then process it
> > reading CQ entries which refer to the messages landing there. When I see a CQ entry
> > with FI_MULTI_RECV in its flags I re-post that same multi-receive buffer with
> > fi_recvmsg() again. Based on reading the man pages, I've put an assertion in my CQ
> > entry processing that if an entry has FI_MULTI_RECV in its flags, that entry must be
> > the last one my fi_cq_read() read. Essentially, this assertion is to confirm my
> > understanding that nothing can be placed in the multi-receive buffer after the
> > releases it.
> This should be the case. What version of the library are you using?
> Are you posting more than 1 multi-recv buffer? There were issues in older code where
> providers were reposting multi-recv buffers to the end of the receive queue. So, if
> there were 2 buffers posted, received data would intermingle between them. (Data
> ordering was unaffected).
In addition, there was also an issue in the multi-recv posting path not handling unexpected messages correctly. I don't remember which version of libfabric correct the above issues, but they should be fixed in the upstream code.
More information about the Libfabric-users