[openib-general] Re: RDMA Generic Connection Management

[Roland Dreier]

    Thomas> Well, you're saying somebody has to do it, right? Is it
    Thomas> easier to fob this off to upper layers that (frankly)
    Thomas> don't care what hardware they're talking to!? This means
    Thomas> we have N copies of this, and N ways to do it. Talk about
    Thomas> cacheline pingpong.

Upper layers have the luxury of being able to do this at a
per-connection level, can sleep, etc.  If we push it down into the
verbs, then we have to do it in every verbs call, including the fast
path verbs call.  And that means we get into all sorts of crazy code
to deal with a device disappearing between a consumer calling
ib_post_send() and the core code being entered, etc.

Right now we have a very simple set of rules:

  An upper level protocol consumer may begin using an IB device as
  soon as the add method of its struct ib_client is called for that
  device.  A consumer must finish all cleanup and free all resources
  relating to a device before returning from the remove method.

  A consumer is permitted to sleep in its add and remove methods.

 - R.