[ofa-general] Re: [openib-general] [RFC] [PATCH] ib_cache: do not mask upper bit when searching for a pkey

[Or Gerlitz]

On 2/26/07, Sean Hefty <sean.hefty at intel.com> wrote:
> I think the following patch would make ipoib spec compliant.
> ib_find_cached_pkey is called by ib_cm, rdma_cm, ib_srp, and ib_ipoib.
> I'm not certain what this change would do to SRP, but the ib_cm and
> rdma_cm look okay, given that non-reversible paths aren't supported
> yet anyway.

Sean,

As Moni stated, we need this functionality and among other scenarions,
the use case I have mentioned over this discussion was of an I/O
target being a full member of a partition where the initiators
connected to it being partial members - since they need not and should
not talk among themselves.

The connection may be implemented over TCP/UDP on top of IPoIB (eg
iscsi / nfs / some cluster file system) or over the RDMA CM and the
VERBS (iSER / rNFS / native implementation of cluster file systems) or
over the IB CM and the VERBS (srp).

For all the above cases expect for SRP IPoIB is used as the ARP
provider and it means that the nodes with the partial membership must
join the "IPv4 broadcast" IB multicast group. This is working fine
with the openib IPoIB and core implementation running against the
Voltaire SA/SM and as Hal commented (Hal - can you verify it? see (*)
below ) also against the open SM/SA. My guess this is also working
fine with TopSpin/Cisco SM/SA.

(*) simply configure the SM to allocate 0xffff (index 0) and 0x8001
(index 1) to node A, then 0x7fff (index 0) and 0x0001 (index 1) to
node B. Now, configure ib0 of both nodes to subnet X, create an 0x8001
ib0 child on both and configure ib0.8001 to subnet Y, make sure you
have pings on both subnets - thanks!

My suggestion is that we act to have the spec changed to match this
real need and not that this code (my guess which is present there from
day one, I guess Roland can tell) would be removed to match the spec.

Or.