[ofa-general] [RFC] [PATCH 0/3] 2.6.22 or 23 ib: add path record cache

[Sean Hefty]

The following set of patches adds a local SA path record cache to the IB stack
(currently based on 2.6.20-rc4).  The cache is derived from the OFED 1.2 local
SA cache patches, with changes based on the last round of feedback and current
Path Forward feature requests:

* InformInfo/Notice support added to ib_sa
Clients may now register to receive SA related events.  The local_sa uses this
to receive notification of GID up/down events in order to keep the cache up to
date.

* Removal of time based cache updates
Cache updates are now driven by local and SA events.  Most module parameters
have been eliminated, and remaining options are exposed through a file system
interface for dynamic control, including the ability to force a cache refresh.

Using a local SA cache we were able to establish all-to-all connections between
1024 processes (about 1 million connections) in about 3 seconds.  Without the
cache, connection time took about a minute, and required a substantial amount of
tuning of timeout values to achieve this.

I've only updated the rdma_cm to use the cache, but similar changes could be
made to SRP and ipoib (which implements its own path record cache).

I would like to get feedback on both the notice and local_sa patches for
inclusion in 2.6.22 or 2.6.23 (if 2.6.22 is not possible).

Signed-off-by: Sean Hefty <sean.hefty at intel.com>