[openib-general] [PATCH]proposal for enabling partial ports on HCA

[Roland Dreier]

    Shirley> I don't think we handle "half-usable" devices here. We
    Shirley> treat each port as an individual "device" in many layers,
    Shirley> ports to ports are independent. For each HCA which could
    Shirley> be as many as 256 ports, I think it makes more sense to
    Shirley> handle per port, not per HCA device based.

The problem with this view is that the HCA is really the fundamental
object in the model described in the IB spec.  Most transport
resources are attached to an HCA, not a port.  In fact, with APM, a QP
might be attached to two different ports at the same time.

    Shirley> Second, The IB SW stack shouldn't prevent any
    Shirley> implementation from handling later ports becoming
    Shirley> usable. The SW implementation should support all kinds of
    Shirley> HCA implementations. Doesn't matter if it is IBM HCAs or
    Shirley> HCAs from other vendors in the future.

I definitely don't want to block support for IBM HCAs.  However, at
the same time I don't want to make the IB stack more complex, more
error-prone, etc. just to work around what I would argue is a bug in
your firmware.

    Shirley> Third ib_cache & ib_ipoib implmentation actually allow
    Shirley> "half-usable" devices. It allows other ports initializing
    Shirley> while one port has errors.

It seems cache.c actually bails out if it fails to allocate space for
one HCA port.  IPoIB does indeed proceed even if one port fails, but
that's more because there's no real reason to bail out halfway rather
than wanting to support half-usable devices.

I don't object much to making layers that really are per-port work
that way.  What worries me is trying to fix everything to work sanely
with individual ports becoming usable or unusable after an HCA has
been attached to the system.

I guess we'll have to wait and see how convincing your patches are.

 - R.