[ofa-general] IPOB CM (NOSRQ) [PATCH V9] patch

[Roland Dreier]

 > And to re-start this discussion, I think we should separate the
 > maximum number of QPs from whether we use SRQ, and let the QP type
 > (UD, UC, RC) be  controllable.  Smaller clusters may perform better
 > without using SRQ, even if it is available.  And supporting UC versus
 > RC seems like it should only take a few additional lines of code.

Actually supporting UC is trickier than it seems, at least for the SRQ
case, since attaching UC QPs to an SRQ requires that the IB spec be
extended to allow that (and also define some semantics for how to
handle messages that encounter an error in the middle of being
received, after a work request has been taken from the SRQ).

 > I agree with Roland that we need to come up with the correct user
 > interface here, and I'm not convinced that what we have is the most
 > adaptable for where the code could go.  What about replacing the 2
 > proposed parameters with these 3?
 > 
 > qp_type - ud, uc, rc
 > use_srq - yes/no (default if available)
 > max_conn_qp - uc or rc limit

I don't think we want the qp_type to be a module parameter -- it seems
we already have ud vs. rc handled via the parameter that enables
connected mode, and if we want to enable uc we should do that in a
similar per-interface way.

Similarly if there's any point to making use_srq something that can be
controlled, ideally it should be per-interface.  But this could be
tricky because it may be hard to change at runtime.

(Ideally max_conn_qp would be per-interface too but that seems too
hard as well)

I do agree that the memory limit just seems arbitrary and we can
probably do away with that.

 - R.