[ofa-general] RE: QoS for iSER
Kanevsky, Arkady
Arkady.Kanevsky at netapp.com
Wed Nov 14 09:22:01 PST 2007
what happens when multiple apps runs on the same server?
Arkady Kanevsky email: arkady at netapp.com
Network Appliance Inc. phone: 781-768-5395
1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195
Waltham, MA 02451 central phone: 781-768-5300
> -----Original Message-----
> From: Hal Rosenstock [mailto:hrosenstock at xsigo.com]
> Sent: Wednesday, November 14, 2007 8:18 AM
> To: Yevgeny Kliteynik
> Cc: gdror at mellanox.co.il; general at lists.openfabrics.org
> Subject: Re: [ofa-general] RE: QoS for iSER
>
> On Wed, 2007-11-14 at 11:11 +0200, Yevgeny Kliteynik wrote:
> > Hal Rosenstock wrote:
> > > Or,
> > >
> > > On 11/13/07, Or Gerlitz <ogerlitz at voltaire.com> wrote:
> > >> Yevgeny,
> > >>
> > >> iSER (as you can learn from doing a grep) is using the
> RDMA-CM TCP
> > >> port space as does RDS. The RDMA-CM signature is
> something which I
> > >> am sure exists, you can look on the RDMA-CM IB spec
> Annex to see if
> > >> such thing indeed exist or I am wrong.
> > >
> > > Did you really look at the annex for this ?
> > >
> > >> The TCP port is the 16 bit port portion of the ip:port address
> > >> provided by a ULP that uses the RDMA-CM to rdma_resolve_addr(),
> > >> again the annex explained how the port is embedded into
> the SID, I
> > >> don't remember the location within the 64 bit string.
> > >
> > > It's in the low 16 bits (bytes 6-7) of the SID as the
> annex indicates.
> > >
> > >> Or.
> > >>
> > >> -------- Original Message --------
> > >> Subject:
> > >> Re: QoS for iSER
> > >> Date:
> > >> Mon, 12 Nov 2007 11:41:43 +0200
> > >> From: Yevgeny Kliteynik <kliteyn at dev.mellanox.co.il>
> > >>
> > >> Hi Erez,
> > >>
> > >> Erez Zilber wrote:
> > >>> to create the SID, the rdma cm combines
> > >>>
> > >>> 1) the port space
> > >> What is the port space for iSER?
> > >> For SDP it's 0x10000 - 0x1FFFF.
> > >> For RDS it's 0x1060000 - 0x106FFFF
>
> I presume this is just saying RDS uses IP protocol TCP and
> there is no well known port (e.g. uses dynamic ports). So how
> do you know ahead of time which port ?
>
> > >> For iSER it's ...?
> > >
> > > These numbers are too large for just "port space".
> > >
> > > iSER SID is 0x000000000106035c
> > >
> > > in your nomenclature, I guess 0x106035c
> > >
> > > 01 says RDMA aware ULP service ID range
> > > 06 says IP protocol is TCP
> > > 0x035c (port 860) is the well known TCP port for iSCSI
> >
> > Thanks, that is just what I needed.
> > I'm preparing a (very) simplified interface for defining QoS policy.
> > I'm adding an additional section in QoS policy file, where an admin
> > will be able to configure QoS per ULP or per application w/o going
> > into too many details.
> > Here's the example of what I have in mind:
> >
> > qos-ulps
> > default : 0 #default SL
> > sdp, port 10000-20000 : 2
> > sdp : 0 #default SL for SDP
> > rds, port 25000 : 2 #SL for RDS when
> destination port is 25000
>
> Isn't there a chicken and egg problem here with this ? How do
> you know port 25000 will be assigned "in advance" ?
>
> > rds, : 0 #default SL for RDS
>
> I don't see how RDS can work separate from other CMA based
> protocols which use dynamic ports.
>
> > iser *??????* : 4 #SL for iSER
> > ipoib, pkey 0x0001 : 5 #SL for IPoIB on
> partition with pkey 0x0001
> > ipoib : 6 #default IPoIB
> partition - pkey=0x7FFF
> ...
> > end-qos-ulps
> >
> > This syntax is possible only if there are well known facts
> such as SDP
> > service ID, in which case admin will be able to just state "sdp:
> > <sl>", and OpenSM will (internally) generate relevant matching rule
> > and QoS level based on this known service ID.
> >
> > So back to iSER:
> >
> > Can I assume that the target port for iSER will always be
> 860, hence
> > the iSER service ID will always be 0x000000000106035c?
>
> In terms of iSER, I was only commenting on what the spec
> says. I did not verify its operation in terms of the code.
> Does the code follow the spec ?
>
> -- Hal
>
> > Or perhaps I can do it similar to SDP, where there is an option to
> > specify the port ranges along with the ULP name (SDP):
> > - if administrator only specifies "iser", I can assume that
> > the service ID is default 0x000000000106035c
> > - if administrator only specifies "iser" and ports, OpenSM
> > will build service ID based on a well known prefix
> > (0x000000000106pppp) where the last 4 hex digits are target
> > port number
> >
> > Keep in mind that if this doesn't look too flexible and
> doesn't cover
> > all the cases, there's always the rest of the QoS policy
> file with all
> > the advanced configuration.
> >
> > -- Yevgeny
> >
> > > -- Hal
> > >
> > >>> 2) the rdma cm signature
> > >> Do you mean something iSER-specific, or just the way the
> cm builds
> > >> the service ID out of port space and tcp port?
> > >> Can you give an example?
> > >>
> > >>> 3) the destination tcp port provided to rdma_resolve_addr
> > >> I guess that tcp port is in the lower 4 nibs of the service ID,
> > >> similar to SDP. Right?
> > >> -- Yevgeny
> > >>
> > >> _______________________________________________
> > >> general mailing list
> > >> general at lists.openfabrics.org
> > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> > >>
> > >> To unsubscribe, please visit
> > >> http://openib.org/mailman/listinfo/openib-general
> > >>
> > >
> >
> > _______________________________________________
> > general mailing list
> > general at lists.openfabrics.org
> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >
> > To unsubscribe, please visit
> > http://openib.org/mailman/listinfo/openib-general
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit
> http://openib.org/mailman/listinfo/openib-general
>
More information about the general
mailing list