[openib-general] [PATCH 0/4] Dispatch communication relatedevents to the IB CM

[Sean Hefty]

>I sometimes see the COMM_EST event on my (NFS) QP event handler. I
>assumed that this was occurring due to a race condition between the CM
>event handler thread (processing the ESTABLISHED event that moves the QP
>--> RTS), and my I/O tasklets and callbacks. Since my I/Os are handled
>on interrupt/tasklet context, the response to the request (posting a WR
>to the SQ) was happening before the QP was moved to RTS in the CM
>thread. This didn't actually cause a problem since the IB driver (mthca)
>seemed to simply queue up, but not transmit the SQ WR until the
>QP-->RTS.
>
>All that said... I have a few questions:
>
>- Does any of the above make sense?

In the general sense, this is correct.

The COMM_EST event is generated when a receive completion occurs on a QP that is
in the RTR state - as opposed to RTS state.  Typically, a QP is transitioned
from RTR to RTS when an RTU message is processed by the CM.  Since the client
side of the connection can send data immediately after sending an RTU, there's
the potential for the event.  The user data sent over the connection simply has
to be received at the QP before the RTU can be processed.

Also note that the RTU is sent over an unreliable QP, so could be dropped.

>- Do _all_ IB devices (not just mthca) queue WR posted on the SQ when
>the QP is in RTR or can they generate an error if I don't wait until the
>QP --> RTS to reply to a request.

All IB devices should allow posting of WR to the SQ while in the RTR state.

>- Given a _very_ brief review of the patch, it looks like I don't need
>to to change what I'm doing now (nothing). True?

Correct.  The patch fixes what I would consider to be a small window: when user
data is received on a QP, but the RTU is continually lost.

>- Will path migration now be handled automagically in the IB CM?

Not yet.  A user can load an alternate path in the IB CM, but there's still no
mechanism in place for the IB CM to switch from using an old path to a new one
when sending CM messages.  The QP itself will fail-over if an alternate path is
loaded, but the CM messages will continue to use the old path.  This patch
simply extends the framework that is there, but additional coding is needed.

- Sean