[openib-general] Re: ipoib multicast timeouts

[Roland Dreier]

    Sean> The multicast module handles multiple join/leave requests to
    Sean> the same multicast group.  Because multiple join requests
    Sean> can occur simultaneously, SA query timeouts are controlled
    Sean> by module parameters.  Ipoib issues join requests with an
    Sean> exponentially increasing timeout.

The exponential backoff is needed because otherwise large clusters can
get into a situation where the SM is overwhelmed by IPoIB join
requests right after bringing up the fabric, and every node keeps
timing out its join request and retrying quickly, and no progress gets
made because the replies that the SM does generate are for requests
that have already timed out.

    Sean> Some possibilities:

    Sean> 1. Do not migrate ipoib to use the multicast module.

Seems bad because we lose reference counting for IPoIB MCGs.

    Sean> 3. Currently, ipoib backoff is used both as the timeout for
    Sean> a single request and the time between retries.  Instead, use
    Sean> it only the time between retrying requests.

This seems fine.  IPoIB uses the time for both because it's convenient
to implement things by simply waiting for a response and/or timeout,
and then immediately doing the next thing.  So this way saves IPoIB
from having to implement some more code to wait from a timeout event
until the next retry.  But if it's in an external module anyway,
that's fine by me.

 - R.