[openib-general] IP addressing on InfiniBand networks

[David M. Brean]

Hello,

Caitlin Bestler wrote:

>On 6/28/05, Roland Dreier <rolandd at cisco.com> wrote:
>  
>
>>    James> First off, here [are the] requirement we are trying to satisfy:
>>
>>    James>    On the passive side of a connection, a InfiniBand kDAPL
>>    James> provider must determine a source IB address for an
>>    James> InfiniBand connection request.  This information can be
>>    James> obtain by a kDAPL consumer either in the
>>    James> DAT_CONNECTION_REQUEST_EVENT's dat_cr_arrival_event_data or
>>    James> dat_cr_query()'s dat_cr_param structure.
>>
>>    James>    By interoperable, we mean that the solution must not
>>    James> introduce a non-standard protocol or force ULPs using kDAPL
>>    James> to perform special operations when using an InfiniBand
>>    James> network.
>>
>>Since these two points are mutually contradictory -- the IB
>>communication management protocol does not carry enough information
>>for a connection request to be mapped uniquely back to a source
>>address -- we need to figure out which one to drop.
>>
>>I would argue in favor of the solution selected by SDP: when defining
>>the binding of an abstract protocol to the IB transport, put the
>>source and destination IP addresses in the IB-specific connection
>>setup messages.
>>
>> - R.
>>    
>>
>
>The remote identification is a service being provided *to* the ULP.
>SDP, or any other application, can provide whatever additional
>information desired. The "IA Address" is as authenticated as
>the local network management allows it to be. Private data 
>exchanged by applications is outside the view of the network
>administrator and therefore can never  be authenticated in 
>the same way.
>
>For NFSoRDMA, for example, "authenticating" a remote 
>peer based upon an application supplied field in the private
>data is obviously inappropriate.
>
>  
>

In proposals that I've seen to use the CM private-data area, the 
information exchanged for authenticating the remote peer is performed by 
the provider, not the ULP.  The DAPL provider is responsible for 
figuring out how to handle IA addresses.  I think this operation is 
consistent with your statements above.

However, I'm aware of one issue.  I'm told that the DAPL implementation 
already uses the CM private-data area to enable ULPs to exchange 
information.  As a result, there isn't any space left over for a 
SDP-like Hello messages that could be exchanged by the providers.  
Reducing the space available to DAPL ULPs to make room for the Hello 
message may be a change that is visible to ULPs even though this is not 
an API change.

>The GID that InfiniBand does supply *does* fully qualify as
>an IP Address, and can be the address supplied as the
>"remote address" even *if* there are additional methods 
>of translating IPv4 addresses to GIDs (or even IPv6, but
>why you would want to *translate* an IPv6 address to
>a GID rather than just using the GID *as* an IPv6
>address is beyond me).
>
>  
>

I recommend that the mechanism that is chosen be capable of supporting 
all three cases.

Furthermore, I think the impact on IP administrative tools needs to be 
assessed when using IB GIDs as IPv6 addresses.  For example, should it 
be possible to use ifconfig(1m) to assign an IPv6 address to the GID on 
an IB port?  What is the observed behavior of IP administrative tools 
when one of the "additional properties / restrictions defined within 
IBA" related to GID assignment is encountered?  What interfaces are 
required in the SM to enable coordination of IB GIDs with IP layer 
infrastructure?  I think this is all doable, but a bit of work.

I think that we should view the ability to use GIDs as IPv6 addresses as 
an optimization, not a fundamental function of the mechanism.  
Deployment of the optimization can then occur later.

-David

>To clarify: DAT *does* fully support the use of GIDs as
>IA Addresses, however it assumes that link local addresses
>will not be presented to the Consumer (at least when the
>host is attached to more than one subnet). It assumes
>that they will be at least upgraded to site-local.
>
>The IA Address can be easily sub-divided into IPv4
>addresses that need translation, IPv6 addresses that
>need translation and IPv6 addresses that are also GIDs
>and therefore do not need translation.
>
>The existence of the alternate solutions was largely
>driven by the need to deploy early solutions that were
>*not* integrated with the OS naming system, and therefore
>could not assume that the OS already knew what to do
>with a GID.
>
>That is no longer a problem.
>
>Therefore there is no need to change any of the DAT
>semantics. They are adequate as they are, and in
>particular there is no need to eliminate reporting
>of remote peer addresses -- something that is both
>easy and useful for IP networks. And a feature that
>is already in use.
>  
>