[openib-general] design for communication established affiliated asynchronous event handling

[Rimmer, Todd]

> -----Original Message-----
> From: openib Sean Hefty
> Sent: Wednesday, June 28, 2006 7:24 PM
> 
> Roland Dreier wrote:
> >>I suggest the following design: the CMA would replace the event
handler
> >>provided with the qp_init_attr struct with a callback of its own and
> >>keep the original handler/context on a private structure.
> 
> I should also point out that the proposed design will not work for
> userspace.
> I'm hesitant to make this change until a solution for userspace can
also
> be
> found, in the hope that a common fix can be shared.
> 
> - Sean

The approach we took in our proprietary stack was to provide a verbs
driver interface for the CM to register itself with the verbs driver. 

The CM would open the CA, provide its async event callback routine and
perform a special register_cm() verbs call.  Of course most CM traffic
would occur on the GSI QP, so this open CA instance was only for this
purpose.  This special verb was only available in kernel space (avoiding
security issue of application stealing CM interface and because our CM
was in the kernel anyway).

When the CA got an Async Event for a Communication Established event, it
would deliver it to both the CM (regardless of which QP it was for) and
to the open instance owning the QP.  All other async events were only
delivered to the appropriate open instance.

This put the handling in the kernel and at a low level where it would
not impact handling of other async events and avoided complications of
user vs kernel async event filters.

Depending on the design of APM, the CM might also be interested in APM
related Async Events (in our design the application had an opportunity
to select a new alternate path, so it was more appropriate to let the
ULP handle these events directly).

Todd Rimmer