[libfabric-users] Tagged message question

[Hefty, Sean]

> 	Who provided the memory that the received message went into between being
> received and my posting the recv buffer? Did libfabric supply some memory buffer 'on
> the fly' that the tagged message went into?

Adding to Chris' comments, these are known as 'unexpected messages'.  They are a common MPI problem and nearly impossible to avoid, even for well-designed apps.

Most commonly available hardware lack the ability to perform tag matching.  As a result, rxm buffers all receives before deciding how to process it.  If you're using RDM endpoints, there's really no additional cost for using tagged transfers, versus untagged.

If the transfer is large, it won't complete immediately and the buffering will occur on the send side.  Only the tagged information will be transferred.  Once the receiver posts a receive with the correct tag, the data will be retrieved.

- Sean