[openib-general] Re: OpenIB Thread Usage

[Roland Dreier]

    Sean> I tried to keep the MAD layer from knowing about completion
    Sean> threads to make it easier to change it later.  I think once
    Sean> we get to some CM performance testing, we can try adjusting
    Sean> the threading model to gives us the best performance and
    Sean> scalability: one per port, one per CPU shared across all
    Sean> ports, one per system, dynamically allocated threads, etc.

I think the CM ends up needing its own set of workqueues so that it
can queue MAD processing along with "time wait" events etc.  Also we
don't want the CM to block general MAD processing while it waits for
things like QP modify.

    Sean> Right now, I'm not sure what sort of performance hit we'll
    Sean> see by having additional idle threads when MAD traffic is
    Sean> low.

idle threads have pretty minimal impact beyond the memory they use.
However on say a 512 CPU box with 6 HCAs, we would create 6000+ kernel
threads, which seems pretty excessive.

 - R.