[ofa-general] [PATCH 3/4] rdma/cm: add ability to specify typeof service

[Eitan Zahavi]

Hi Or, Sean,

SID and QoSClass have a different purpose:
* SID enables the administrator to set the QoS attributes without any
application change. 
   (e.g. a simple rule like all SSH should have lowest priority)
* QoS Class: enables the application to request QoS class on a
per-socket manner. 
   It requires the application to know what it is doing and the
administrator to trust it.

It is not by chance that the two options are there in the spec. 
Different communities/installations have different requirements
regarding "who controls" the policy.

For OFA development I think that providing SID should be done by default
CM based traffic: 
The SID is a mandatory for making a connection and thus I do not see any
reason not to provide it in the PathRecord query.

Supporting QoS-Class is a compatibility feature supporting application
that set the TOS socket option to obtain TOS aware service over IB.


Eitan

Eitan Zahavi
> -----Original Message-----
> From: general-bounces at lists.openfabrics.org 
> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Or Gerlitz
> Sent: Tuesday, August 07, 2007 12:17 PM
> To: Sean Hefty
> Cc: OpenIB
> Subject: Re: [ofa-general] [PATCH 3/4] rdma/cm: add ability 
> to specify typeof service
> 
> Sean Hefty wrote:
> > Provide support to specify a type of service for a communication 
> > identifier.  A new function call is used when dealing with IPv4 
> > addresses.  For IPv6 addresses, the ToS is specified through the 
> > traffic class and flow label fields in the sockaddr_in6 structure.
> 
> > @@ -1508,13 +1520,26 @@ static int cma_query_ib_route(struct 
> > rdma_id_private *id_priv, int
> 
> > +	comp_mask = IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID |
> > +		    IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH |
> > +		    IB_SA_PATH_REC_REVERSIBLE;
> > +
> > +	if (id_priv->id.route.addr.src_addr.sa_family == AF_INET) {
> > +		path_rec.qos_class = cpu_to_be16((u16) id_priv->tos);
> > +		comp_mask |= IB_SA_PATH_REC_QOS_CLASS;
> > +	} else {
> > +		sin6 = (struct sockaddr_in6 *) 
> id_priv->id.route.addr.src_addr;
> > +		path_rec.flow_label = sin6->sin6_flowinfo;
> > +		path_rec.traffic_class = (u8) 
> (be32_to_cpu(sin6->sin6_flowinfo) >> 20);
> > +		comp_mask |= IB_SA_PATH_REC_FLOW_LABEL |
> > +			     IB_SA_PATH_REC_TRAFFIC_CLASS;
> > +	}
> 
> I think the correct way here, is a SID --> QoS mapping. This 
> allows you to manage and configure QoS in --one-- place, 
> namely the SM/SA and zero effort host side deployment.
> 
> For example clustered database has IB subnet both for the 
> inter RAC IPC and for the RAC nodes storage access, say IPC 
> is RDS and storage is iSER (a similar example from HPC, IPC 
> is MPI and I/O is Lustre)
> 
> So you would need to configure an SID to SL mapping at the SA 
> for each service and you are --done--. Taking this to cross 
> IB subnets, the SID based mapping will also provide the TC 
> and FL for the GRH.
> 
> With the above implementation you would have to change each 
> single ULP to have it use QoS and you have to have a 
> configuration file also at the host side (saying what tos to 
> set through the rdma cm) and not only in the SM.
> 
> Or.
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general
>