[openib-general] [PATCH] [CM] add private data comparison to match REQs with listens

[Caitlin Bestler]

openib-general-bounces at openib.org wrote:
> The following patch modifies the kernel CM API to support
> matching private data in received REQs against listen
> requests.  This allows the CM to support multiple listeners
> on the same service ID if a discriminator is carried in the
> private data.
> 
> This will be used by the CMA to distinguish listen requests
> on the same port, but on different IP addresses.  Updates
> were made to SDP and kDAPL to support this change.
> 

A DAT Consumer does not issue multiple listens for the same
Interface Adapter differentiated on the IP address (source
or destination). Only a single listen per port/service ID
is allowed per Interface Adapter.

The resulting Connection Request events carry the remote
and local addresses to allow the DAT *consumer* to differentiate
based on the address.

There are several factors as to why this differentiation
is best done by the DAT Consumer, and not by the Connection
Manager or DAT Provider:

1) While supporting multiple addresses on the same Interface
   is an extremely common practice, typically it is the *same*
   service being supported for port X no matter what the local
   IP address is.
2) Virtual hosts may support *thousands* of IP addresses, and
   differentiate the content served for each of them. Such a
   large fan out is best supported by the daemon's own tables,
   rather than by general purpose Connection Manager tables.
3) The backlog is best expressed as a common limit (HTTPD will
   accept up to 50 pending connections) rather than a per IP
   address limit (since there are *thousands* of virtual sites,
   the aggregate limit simply dissapears).

Keep in mind that this is site virtualization where the daemon
is aware that it is supporting multiple sites using a *single*
Interface Adapter (HCA/RNIC). Apache is the prime example of 
this strategy.

Xen-style virtualization can have each GOS supplying a totally
different daemon for its virtual Interface Adapter on the same
TCP port/Service ID. But in that case each daemon/GOS thinks 
that it has a distinct device. The listents are segregated by
the virtual device, not by the destination IP address (unless
that is how the virtualizer splits the traffic, but that is
done *before* the Connection Manager is involved).