[openib-general] posting send requests in RTR

[Caitlin Bestler]

Sean Hefty wrote:
>> That assumes that there is any valid reason for an application to
>> post send requests before the connection is established. While there
>> is clearly a need to post receive work requests before the
>> connection is established I cannot think of any reason why an
>> application needs to pre-prime the send queue.
> 
> It's not pre-priming the send queue.  An application could
> pull a completed receive work completion off of a CQ.  The
> receive may very well be a request that requires a response.
> At this point, the connection is obviously established from
> the consumers viewpoint, but not necessarily by the viewpoint
> of the RDMA CM or IB CM.  The response must now be queued.
> 

Or you could have the verbs provider infer that since a receive
work request has completed then the connection is obviously
established, and it could generate the "connection established"
event.

That way the Consumer always gets a "connection established" event.

Alternately, it would be reasonable to simply document that a
receive completion *implied* a connection established event,
and therefore the application could post to the send queue
after it reaped a receive completion (or got a connection
established event).

Most servers only transmit in response to requests received,
so that could actually be an even simpler rule for many server
applications.

The iWARP angle on this question does not deal with queuing
responses, but rather that RDMAC compliant RNICs assume that
the host software is responsible for ensuring that the connection
complies with the MPA requirement that the active side send the
first MPA Frame. If the application posts to the send queue *before*
the Connection Established event, an RDMAC compliant RNIC is likely
to send it anyway, thereby putting the MPA connection at risk. If
that DDP Segment is received by the active side before the MPA
Response there is no guarantee that it will be processed correctly.