[swg] Re: [openib-general] RE: [dat-discussions] socket based connectionmodel for IB proposal - round 3
Renato Recio
recio at us.ibm.com
Wed Nov 16 15:52:00 PST 2005
The SDP port mapper protocol is required, because the well known SDP port
is "not" the targeted port.
In the case of the RDMA Service, the targetted port may very well be a
well known port (e.g. one of the two iSCSI ports), but the only point of
the RDMA Service is to let the listening end know "I want the iSCSI well
known port, that is RDMA aware".
It seems to me that for ULPs that are RDMA aware, there is no need for a
port mapper. However, over IB there is a need to know that 1) its the RDMA
aware service being requested; and 2) to leverage the IP stack, the IP
address info.
More comments bounded by <rr> below.
Thanks,
Renato J Recio
Chief Architect, eServer I/O
IBM Distinguished Engineer
Member IBM Academy of Technology
Tel 512-838-3685, T/L 678-3685
Michael Krause
<krause at cup.hp.co To: "Kanevsky, Arkady" <Arkady.Kanevsky at netapp.com>
m> cc: swg at infinibandta.org, openib-general at openib.org
Subject: Re: [swg] Re: [openib-general] RE: [dat-discussions] socket based
11/16/2005 04:15 connectionmodel for IB proposal - round 3
PM
At 10:22 AM 11/15/2005, Sean Hefty wrote:
Kanevsky, Arkady wrote:
Which entity is responsible to "use" the proposed protocol is
an
interesting one.
I was assuming that this will be CM. After all the proposed
protocol is
CM extension
protocol.
But it can be another entity module between CM and ULP.
The use of a reserved bit in the CM message indicates that the CM
itself needs to set this data. This requires communication between
the CM and IPoIB - in essence making IPoIB part of the CM. Removal
of the reserved bit is what permits another entity between the CM and
the ULP to perform this task.
While it is possible to do wildcarding on the whole SID, I had
not seen
it is used selectively on individual bits of a SID or a port.
This is what is done/needed to support SDP. A simple mask is applied
to the SID before comparing against a local listen.
While SDP does the conversion to IB SID from Ethernet port,
this
proposal
shift the responsibility for port and IP address conversion
from ULP
down.
Ideally, SDP should use this same mechanism, but requires changes to
SDP.
Given SDP is an existing standard and implemented on a variety of
platforms, it would be best to not modify SDP in order to insure
interoperability. Hence, the new annex should only apply to ULP that do
not already communicate the IP address, etc. as part of the CM exchange on
IB and as part of the SDP port mapper on iWARP (the port mapper enables
completely transparent SDP usage underneath an application with no source
code changes required).
<rr> No. Mike, the intent is not to use this protocol for a ULP that is
not RDMA aware and runs over IPoIB. Similarly, the intent is not to use
this protocol fo a ULP that is not RDMA aware, but depends on SDP to help a
little by leveraging RDMA as much as possible without exposing RDMA
primitives (RDMA Write/Read) to the ULP.
Instead, the intent is to use this protocol for a ULP that is RDMA aware.
Think of it as...
IPoIB uses IP stack for port management (reaching well known ports, port
mapping for dynamic ports, ...).
SDP uses the IB SDP Service for managing ULP ports that are not RDMA aware,
but we want to help a little.
We need a similar IB RDMA Service for managing ULP ports that RDMA aware.
I brought this up in the RDMAC many moons ago. It seems to me that for
iWARP the solution is a little more straight forward, because the stack
stays IP through out. That is, if the active side (in IB terms :{) wants
to connect to a well known port on the passive side, the RDMA negotiation
can occur in band (e.g. as it does for iSER). However, given we don't have
the IP stack through out, we need an RDMA service that allows an active
side RDMA aware ULPs to ask the passive side ULP "are you RDMA aware?".
That Service is what is being proposed. <rr>
Protocol version. This mean that in the future if protocol
version will be
bumped up we will have to change the SID on which Consumer
listens on and
requests sent to. Not sure how to do that without changing ULP.
Does not look
like a good idea.
As a note, I'm not saying that I prefer a more complex SID, just that
there is trade-off to be made that could provide for more ULP private
data.
If the version changes, then the addressing has changed. The ULP may
need to change anyway in order to know how to interpret the address.
If they don't care about the protocol version or don't need to know
how to interpret the address, they can still wildcard the version.
IP version. This can be incorporated into SID. But if HCA has
multiple IP addresses
assigned to it the listening point need to specify its IP
address(es).
The current verbs and/or API will have to be changed to support
it.
But if socket is passed to listen on it does have all the
needed info.
Looks fine.
I didn't follow this.
The SDP port mapper protocol for iWARP already solves the multiple IP
address issue. The same concept / principles could be leveraged for IB. I
don't think the verbs need to be changed to support it as it is just
payload within CM messages for IB.
<rr> But they can't. Because for IB, we will still need a Service ID (see
CM spec chapter) and the SDP Service ID is not applicable (e.g. we will not
be running SDP between iSCSI and iSER). <rr>
DAPL APIs (uDAPL and kDAPL) does not expose local IP address
for listen
point.
An additional API can be added to support passing local socket
to listen
on
instead of Connection Qualifier. Since it is addition no
backwards
compatibility issues.
The current ULPs/Apps will still use "the default API address"
and the
protocol assigned SID as connection qualifier.
The issue with DAPL is that it assumes that addressing has been
resolved to a specific device before communication between the client
and server have even occurred. I.e. the server must be clairvoyant
and know which device a connection request will be received on.
Likewise, a client must assume which local device is needed to
connect to a given remote address.
The nice thing about the SDP port mapper is that it allows the server and
client to dynamically determine the resolution prior to connection
establishment as well as gauge how many resources, preferred CA / port,
etc. to extend to the client.
<rr> Read my comments above. <rr>
Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20051116/8c29d78d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20051116/8c29d78d/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ecblank.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20051116/8c29d78d/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic08516.gif
Type: image/gif
Size: 1255 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20051116/8c29d78d/attachment-0002.gif>
More information about the general
mailing list