[dat-discussions] [openib-general] [RFC] DAT2.0immediatedataproposal

[Roland Dreier]

    Arlin> A very latency sensitive application that requires
    Arlin> immediate notification of RDMA write completion on the
    Arlin> remote node without ANY latency penalties associated with
    Arlin> combining operations, HCA priority rules across QPs, wire
    Arlin> congestion, etc. An application that has no requirement for
    Arlin> messaging outside of remote rdma write completion
    Arlin> notifications. The application would not have to register
    Arlin> and manage additional message buffers on either side, we
    Arlin> can just size the queues accordingly and post zero byte
    Arlin> messages. We need something that would be equivelent to
    Arlin> setting there polling on the last byte of inbound
    Arlin> data. But, since data ordering within an operation is not
    Arlin> guaranteed that is not an option. So, rdma with immediate
    Arlin> data is the most optimal and simplistic method for
    Arlin> indication of RDMA-write completion that we have available
    Arlin> today. In fact, I would like to see it increased in size to
    Arlin> make it even more useful.

Hmm.  Can you put a number on how much better RDMA write with
immediate is on current HCA hardware?  How does using the underlying
OpenIB verbs ability to post a list of work requests compare (ie
posting an RDMA write followed by a send in one verbs call)?
Maybe "post multiple" is a better direction for DAT.

 - R.