[ofa-general] RE: QoS for iSER

Kanevsky, Arkady Arkady.Kanevsky at netapp.com
Wed Nov 14 09:22:01 PST 2007


what happens when multiple apps runs on the same server?

Arkady Kanevsky                       email: arkady at netapp.com
Network Appliance Inc.               phone: 781-768-5395
1601 Trapelo Rd. - Suite 16.        Fax: 781-895-1195
Waltham, MA 02451                   central phone: 781-768-5300
 

> -----Original Message-----
> From: Hal Rosenstock [mailto:hrosenstock at xsigo.com] 
> Sent: Wednesday, November 14, 2007 8:18 AM
> To: Yevgeny Kliteynik
> Cc: gdror at mellanox.co.il; general at lists.openfabrics.org
> Subject: Re: [ofa-general] RE: QoS for iSER
> 
> On Wed, 2007-11-14 at 11:11 +0200, Yevgeny Kliteynik wrote:
> > Hal Rosenstock wrote:
> > > Or,
> > > 
> > > On 11/13/07, Or Gerlitz <ogerlitz at voltaire.com> wrote:
> > >> Yevgeny,
> > >>
> > >> iSER (as you can learn from doing a grep) is using the 
> RDMA-CM TCP 
> > >> port space as does RDS. The RDMA-CM signature is 
> something which I 
> > >> am sure exists, you can look on the RDMA-CM IB spec 
> Annex to see if 
> > >> such thing indeed exist or I am wrong.
> > > 
> > > Did you really look at the annex for this ?
> > > 
> > >> The TCP port is the 16 bit port portion of the ip:port address 
> > >> provided by a ULP that uses the RDMA-CM to rdma_resolve_addr(), 
> > >> again the annex explained how the port is embedded into 
> the SID, I 
> > >> don't remember the location within the 64 bit string.
> > > 
> > > It's in the low 16 bits (bytes 6-7) of the SID as the 
> annex indicates.
> > > 
> > >> Or.
> > >>
> > >> -------- Original Message --------
> > >> Subject:
> > >> Re: QoS for iSER
> > >> Date:
> > >> Mon, 12 Nov 2007 11:41:43 +0200
> > >> From: Yevgeny Kliteynik <kliteyn at dev.mellanox.co.il>
> > >>
> > >> Hi Erez,
> > >>
> > >> Erez Zilber wrote:
> > >>> to create the SID, the rdma cm combines
> > >>>
> > >>> 1)       the port space
> > >> What is the port space for iSER?
> > >> For SDP it's 0x10000 - 0x1FFFF.
> > >> For RDS it's 0x1060000 - 0x106FFFF
> 
> I presume this is just saying RDS uses IP protocol TCP and 
> there is no well known port (e.g. uses dynamic ports). So how 
> do you know ahead of time which port ?
> 
> > >> For iSER it's ...?
> > > 
> > > These numbers are too large for just "port space".
> > > 
> > > iSER SID is 0x000000000106035c
> > > 
> > > in your nomenclature, I guess 0x106035c
> > > 
> > > 01 says RDMA aware ULP service ID range
> > > 06 says IP protocol is TCP
> > > 0x035c (port 860) is the well known TCP port for iSCSI
> > 
> > Thanks, that is just what I needed.
> > I'm preparing a (very) simplified interface for defining QoS policy.
> > I'm adding an additional section in QoS policy file, where an admin 
> > will be able to configure QoS per ULP or per application w/o going 
> > into too many details.
> > Here's the example of what I have in mind:
> > 
> >    qos-ulps
> >        default                       : 0 #default SL
> >        sdp, port 10000-20000         : 2
> >        sdp                           : 0 #default SL for SDP
> >        rds, port 25000               : 2 #SL for RDS when 
> destination port is 25000
> 
> Isn't there a chicken and egg problem here with this ? How do 
> you know port 25000 will be assigned "in advance" ?
> 
> >        rds,                          : 0 #default SL for RDS
> 
> I don't see how RDS can work separate from other CMA based 
> protocols which use dynamic ports.
> 
> >        iser    *??????*              : 4 #SL for iSER
> >        ipoib, pkey 0x0001            : 5 #SL for IPoIB on 
> partition with pkey 0x0001
> >        ipoib                         : 6 #default IPoIB 
> partition - pkey=0x7FFF
>        ...
> >    end-qos-ulps
> > 
> > This syntax is possible only if there are well known facts 
> such as SDP 
> > service ID, in which case admin will be able to just state "sdp: 
> > <sl>", and OpenSM will (internally) generate relevant matching rule 
> > and QoS level based on this known service ID.
> > 
> > So back to iSER:
> > 
> > Can I assume that the target port for iSER will always be 
> 860, hence 
> > the iSER service ID will always be 0x000000000106035c?
> 
> In terms of iSER, I was only commenting on what the spec 
> says. I did not verify its operation in terms of the code. 
> Does the code follow the spec ?
> 
> -- Hal
> 
> > Or perhaps I can do it similar to SDP, where there is an option to 
> > specify the port ranges along with the ULP name (SDP):
> >   - if administrator only specifies "iser", I can assume that
> >     the service ID is default 0x000000000106035c
> >   - if administrator only specifies "iser" and ports, OpenSM
> >     will build service ID based on a well known prefix
> >     (0x000000000106pppp) where the last 4 hex digits are target
> >     port number
> > 
> > Keep in mind that if this doesn't look too flexible and 
> doesn't cover 
> > all the cases, there's always the rest of the QoS policy 
> file with all 
> > the advanced configuration.
> > 
> > -- Yevgeny
> > 
> > > -- Hal
> > > 
> > >>> 2)       the rdma cm signature
> > >> Do you mean something iSER-specific, or just the way the 
> cm builds 
> > >> the service ID out of port space and tcp port?
> > >> Can you give an example?
> > >>
> > >>> 3)       the destination tcp port provided to rdma_resolve_addr
> > >> I guess that tcp port is in the lower 4 nibs of the service ID, 
> > >> similar to SDP. Right?
> > >> -- Yevgeny
> > >>
> > >> _______________________________________________
> > >> general mailing list
> > >> general at lists.openfabrics.org
> > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> > >>
> > >> To unsubscribe, please visit 
> > >> http://openib.org/mailman/listinfo/openib-general
> > >>
> > > 
> > 
> > _______________________________________________
> > general mailing list
> > general at lists.openfabrics.org
> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> > 
> > To unsubscribe, please visit 
> > http://openib.org/mailman/listinfo/openib-general
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general
> 



More information about the general mailing list