[openib-general] posting send requests in RTR

[Sean Hefty]

>That assumes that there is any valid reason for an application to
>post send requests before the connection is established. While there
>is clearly a need to post receive work requests before the connection
>is established I cannot think of any reason why an application needs
>to pre-prime the send queue.

It's not pre-priming the send queue.  An application could pull a completed
receive work completion off of a CQ.  The receive may very well be a request
that requires a response.  At this point, the connection is obviously
established from the consumers viewpoint, but not necessarily by the viewpoint
of the RDMA CM or IB CM.  The response must now be queued.

I believe that the problem can be limited under the following application
conditions:

1. The application uses the CQ with different QPs.
2. The application is on the passive (server) side of the connection.
3. The active (client) side sends a request to the server.

Even combined these conditions could easily occur.

IMO, the question is do we pass this problem to the applications to deal with,
or try to handle transparently it under verbs.  If we try to handle it under
verbs, can it be done in one place?  How much would such checks affects the
performance of post send operations?  And how would immediate or other errors be
handled when posting queued sends?

My personal take at the moment is to let the ULPs handle the problem.

- Sean