[openib-general] [RFC] [uCM] proposed API changes

[Sean Hefty]

>Serialize events for user-mode cm_ids, and allow the user client to set the
>context from their REQ handler.  The latter is probably pretty easy to do, but
>in and of itself doesn't solve the problem with the out-of-order events and
>races between setting the context and receiving an event.

Callbacks aren't used in usermode to report events.  A user calls get_event,
then put_event when they're done.  Get and put can come in separate threads.
I'm not sure about blocking a user's thread until put is called.

Setting the context is actually a little complex than the client setting a field
in a data structure.  To avoid searching, the userspace cm_id needs to be
created, then stored with the kernel cm_id, so that it can be returned with
future events.

>A search on a REJ isn't a big deal - it should be a rare case as it will only
>occur if the remote side times out or aborts.  A client could ignore the REJ
>because sending the REP will fail if a REJ was received.

I agree that a search here isn't a big deal.  But if the REJ reaches userspace
first, then it won't find a cm_id.  If it creates one (in order to report the
REJ to the user), then all REQs now need to search.  And I'm hesitant to ignore
the REJ, but that may be the best option if a cm_id isn't found.

>If you allow the user to target a get_event call to a specific cm_id this
>problem goes away.  If the user issues multiple requests against the same
>cm_id,
>they need to be ready to deal with out-of-order event reporting.  This also
>solves the context issue, since the REJ won't be reported until the user
>requests an event from that specific cm_id.

This is an option, but a significant change from the existing implementation.

- Sean