[openib-general] Semantics of transport neutral connection establishment (was Re: [swg] Re: private data...)

[Caitlin Bestler]

[caitlin]
Comments inline
[/caitlin]


-----Original Message-----
From: Tom Tucker [mailto:ttucker at es335.com]
Sent: Thu 10/20/2005 9:26 PM
To: Caitlin Bestler
Cc: Sean Hefty; Fab Tillier; swg at infinibandta.org; openib-general at openib.org
Subject: Re: [openib-general] Semantics of transport neutral connection
establishment (was Re: [swg] Re: private data...)
 

I think this is a useful discussion, however, I would point out that
some of the information being exchanged doesn't have to be in the
private data. It could be exchanged in an untagged send/recv after the
connection is established; which has the benefit of allowing the
application to use an arbitrarily large chunk of data to authenticate
and authorize the remote peer instead of trying to boil the ocean in 64B
of data.

It might be better to only solve the core problem ... which I think is
identifying the service and QP configuration. 

On Thu, 2005-10-20 at 14:58 -0700, Caitlin Bestler wrote:
> I believe a review of what the implementer of a transport neutral
> daemon for an RDMA protocol would be expecting from a Connection
> Management service:
> 
> -- It expects that it can listen for connection requests on a specific
>    16-bit port number (with traditional TCP port number semantics) on
>    either a specific IP Address or for all IP Addresses associated with
>    the network device.

I would expand this to say "...or on all interfaces that have an IP
address". From my reading, it seems to me that the current CMA does
this. 

> -- It will receive connection requests that were initiated by active peers
>    that wish to establish a reliable connection for the purpose of 
>    exchanging RDMA messges.
> 
>    This Connection Request will identify:
> 	1) The remote IP Address of the active peer.
>  This will beauthenticated
>          in the sense that the address is known to have more meaning than 
>          just being a value made up by a remote user-mode peer. If it is a
> lie then privileged software is complicit in the lie. The address may be
>          even more authenticated than that.

Are you saying that it should not be possible for a user mode peer to
masquerade as another host? If this is what you're saying, then I don't
think it is any more secure done in the kernel than in user mode because
the remote peer has no way of knowing where the data was prepared. 

I think that if authentication is the purpose of the remote address,
don't bother. If the active peer needs to be authenticated, do it after
connection establishment when you can exchange signatures of sufficient
size to be useful. 

Am I missing something here?


[caitlin]
I'm merely noting what a TCP daemon is able to assume today as
part of IP connection setup. The remote IP address supplied may
be heavily authenticated (the local network actively prevents IP
spoofing by checking routes, etc.) or next to worthless (the only
guarantee si that the forger had root access on some machine).

Regardless of whether this level of authentication is advisable, it
is *exactly* the assumption that many servers make today. This
includes many NFS configurations. When the network is isolated
from external connections, and the network administrator is 
confident that they control "root" for all machines within the
local network this can be a signifigant level of defense. Within
a corporate intranet, for example, this may be the mechanism
to ensure that marketing does not examine internla engineering
documents. I wouldn't recommend it for protecting HR files, but
it can be quite adequate for many purposes.

More importantly, if this guarantee is not provided then an
explicit warning should be made. For example, unless the
CM header itself is mariked as having IP data in it there is no
way to know that a user mode application simply hasn't made
up an IP address and submitted as part of a normal CM requests
private data.
[/caitlin]



> 	2) The destination IP address that the active peer requested. That
> is, if the network device supports multiple addresses concurrently (as with
> a
>          web farm) the connection request will identify *which* address was
>          specified by the remote active peer.

... and port. I agree this is needed because it is part of the "local
service signature" aka service id aka port number/ip address"

[caitlin]
But the port was implicit in the listen. There is no need for it to 
be part of the Connection Request reported to the listener. If there
is a desire to conserve Service IDs then the wire protocol could certainly
include the TCP port in the CM Private Data. But that would be transparent
to the application.

The transparency to the applicaion is my concern. I'll confess to not 
seeing any great need to avoid allocating 64K Service IDs out of a
64-bit range -- but that's an issue for the IB developers to work out.
[/caitlin]

> 	3) Private Data supplied by the remote peer to establish its
> identity,

IMHO, private data is useless (or at least insecure) for this purpose
for the reasons mentioned above.

>          the required characteristics of the desired connection and/or
other
>          application specific purposes. The private data is supplied prior
to
> 	   connection establishment specifically to enable
> selection/configuration of the RDMA QP. 

I agree that this is a useful purpose for private data.

[caitlin]
Agreed. This is why private data exists. Application developers
SHOULD use it only for this purpose, and use Send/Recv to exchange
other information.

But the problem is once you label it "private data" the application
developers tend to think of it as their data that they can use anyway
they see fit.
[/caitlin]

> Note that on a transport neutral basis the passive
> side application cannot assume that the QP is fully configured to match
> credit requirements of the remote peer -- it must configure QP capacities
> itself.

> -- It will NOT receive connection requests from remote peers seeking to
> connect
>    with similar services based upon streaming socket semantics (SDP or
plain
> TCP).

What specifically are you saying here? That the app won't see the
connect request until after an MPA Start Request has been received?

[caitlin]
Correct, it is not an IP-ssemantics RDMA Connection Request until 
there is no doubt that it is.

Over iWarp/MPA that is established by a valid MPA Request frame.
Over iWARP/SCTP it is established by an association parameter.

For IP-Address-based IB CM that implies that we have to know that
the Private Data actually is in the IP-Address bearing format.

If a stray TCP client who has no idea that they are connecting to an
RDMA daemon initiates a TCP connection that connection will be
handled strictly by the iWARP CM and never reported as an iWARP
connection request to the Consumer.

Similarly, if a stray IB client that has no idea what these "IP semantics"
options are about attempts to connect to a Service that wants the
IP-format CM Private Data the connection request should not be 
reported to the Consumer. Especially not with garbled private data
and addressign data taken from the intended private data.

That would be a change in the smenatics for a transport neutral 
daemon. It is not responsible for figuring out if the client really
meant to connect with it. Sorting out those who do not know how
to speak the relevant Connection Establishment protocol should
remain a responsibiloity of the Connection Manager (for whatever
connection establishment protocol is in use) and not shifted to
the listener.
[/caitlin]

> 
> -- If it so chooses, it may accept the connection request by supplying a
> compatibly
>    configured RDMA QP and response private data.
> -- If it so chooses, it may reject the conneciton request. 

> Many of these requirements point to why the additional data is needed, and
> why taking
> the first N bytes of the existing private data is requried.
> 
> The key requirement that I belive requires that "65th bit" is that a client
> seeking
> a streaming mode daemon cannot initiate a connection with an RDMA mode
daemon
> and 
> start mis-exchanging data.

Are you referring to the MPA Start Key thingy again? I don't think the
IB guys don't have this issue.

[caitlin]
They do. But it's not distinquishing a plain TCP client from an iWARP
client. It's distinquishing a Connection Request from a client who wants
an IP semantics connection from one who is using the base CM protocol.
[/caintlin]