[ofa-general] RE: QoS for iSER

[Hal Rosenstock]

On Wed, 2007-11-14 at 11:11 +0200, Yevgeny Kliteynik wrote:
> Hal Rosenstock wrote:
> > Or,
> > 
> > On 11/13/07, Or Gerlitz <ogerlitz at voltaire.com> wrote:
> >> Yevgeny,
> >>
> >> iSER (as you can learn from doing a grep) is using the RDMA-CM TCP port
> >> space as does RDS. The RDMA-CM signature is something which I am sure
> >> exists, you can look on the RDMA-CM IB spec Annex to see if such thing
> >> indeed exist or I am wrong.
> > 
> > Did you really look at the annex for this ?
> > 
> >> The TCP port is the 16 bit port portion of
> >> the
> >> ip:port address provided by a ULP that uses the RDMA-CM to
> >> rdma_resolve_addr(), again the annex explained how the port is embedded
> >> into the SID, I don't remember the location within the 64 bit string.
> > 
> > It's in the low 16 bits (bytes 6-7) of the SID as the annex indicates.
> > 
> >> Or.
> >>
> >> -------- Original Message --------
> >> Subject:
> >> Re: QoS for iSER
> >> Date:
> >> Mon, 12 Nov 2007 11:41:43 +0200
> >> From: Yevgeny Kliteynik <kliteyn at dev.mellanox.co.il>
> >>
> >> Hi Erez,
> >>
> >> Erez Zilber wrote:
> >>> to create the SID, the rdma cm combines
> >>>
> >>> 1)       the port space
> >> What is the port space for iSER?
> >> For SDP it's 0x10000 - 0x1FFFF.
> >> For RDS it's 0x1060000 - 0x106FFFF

I presume this is just saying RDS uses IP protocol TCP and there is no
well known port (e.g. uses dynamic ports). So how do you know ahead of
time which port ?

> >> For iSER it's ...?
> > 
> > These numbers are too large for just "port space".
> > 
> > iSER SID is 0x000000000106035c
> > 
> > in your nomenclature, I guess 0x106035c
> > 
> > 01 says RDMA aware ULP service ID range
> > 06 says IP protocol is TCP
> > 0x035c (port 860) is the well known TCP port for iSCSI
> 
> Thanks, that is just what I needed.
> I'm preparing a (very) simplified interface for defining QoS policy.
> I'm adding an additional section in QoS policy file, where an admin
> will be able to configure QoS per ULP or per application w/o going
> into too many details.
> Here's the example of what I have in mind:
> 
>    qos-ulps
>        default                       : 0 #default SL
>        sdp, port 10000-20000         : 2
>        sdp                           : 0 #default SL for SDP
>        rds, port 25000               : 2 #SL for RDS when destination port is 25000

Isn't there a chicken and egg problem here with this ? How do you know
port 25000 will be assigned "in advance" ?

>        rds,                          : 0 #default SL for RDS

I don't see how RDS can work separate from other CMA based protocols
which use dynamic ports.

>        iser    *??????*              : 4 #SL for iSER
>        ipoib, pkey 0x0001            : 5 #SL for IPoIB on partition with pkey 0x0001
>        ipoib                         : 6 #default IPoIB partition - pkey=0x7FFF
       ...
>    end-qos-ulps
> 
> This syntax is possible only if there are well known facts
> such as SDP service ID, in which case admin will be able to
> just state "sdp: <sl>", and OpenSM will (internally) generate
> relevant matching rule and QoS level based on this known
> service ID.
> 
> So back to iSER:
> 
> Can I assume that the target port for iSER will always be 860,
> hence the iSER service ID will always be 0x000000000106035c?

In terms of iSER, I was only commenting on what the spec says. I did not
verify its operation in terms of the code. Does the code follow the
spec ?

-- Hal

> Or perhaps I can do it similar to SDP, where there is an option
> to specify the port ranges along with the ULP name (SDP):
>   - if administrator only specifies "iser", I can assume that
>     the service ID is default 0x000000000106035c
>   - if administrator only specifies "iser" and ports, OpenSM
>     will build service ID based on a well known prefix
>     (0x000000000106pppp) where the last 4 hex digits are target
>     port number
> 
> Keep in mind that if this doesn't look too flexible and doesn't
> cover all the cases, there's always the rest of the QoS policy
> file with all the advanced configuration.
> 
> -- Yevgeny
> 
> > -- Hal
> > 
> >>> 2)       the rdma cm signature
> >> Do you mean something iSER-specific, or just the way the cm
> >> builds the service ID out of port space and tcp port?
> >> Can you give an example?
> >>
> >>> 3)       the destination tcp port provided to rdma_resolve_addr
> >> I guess that tcp port is in the lower 4 nibs of the service ID,
> >> similar to SDP. Right?
> >> -- Yevgeny
> >>
> >> _______________________________________________
> >> general mailing list
> >> general at lists.openfabrics.org
> >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>
> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
> >>
> > 
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general