[ofa-general] [PATCH 3/4] rdma/cm: add ability to specify type of service

[Or Gerlitz]

Sean Hefty wrote:
> Provide support to specify a type of service for a communication
> identifier.  A new function call is used when dealing with IPv4
> addresses.  For IPv6 addresses, the ToS is specified through the
> traffic class and flow label fields in the sockaddr_in6 structure.

> @@ -1508,13 +1520,26 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int

> +	comp_mask = IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID |
> +		    IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH |
> +		    IB_SA_PATH_REC_REVERSIBLE;
> +
> +	if (id_priv->id.route.addr.src_addr.sa_family == AF_INET) {
> +		path_rec.qos_class = cpu_to_be16((u16) id_priv->tos);
> +		comp_mask |= IB_SA_PATH_REC_QOS_CLASS;
> +	} else {
> +		sin6 = (struct sockaddr_in6 *) id_priv->id.route.addr.src_addr;
> +		path_rec.flow_label = sin6->sin6_flowinfo;
> +		path_rec.traffic_class = (u8) (be32_to_cpu(sin6->sin6_flowinfo) >> 20);
> +		comp_mask |= IB_SA_PATH_REC_FLOW_LABEL |
> +			     IB_SA_PATH_REC_TRAFFIC_CLASS;
> +	}

I think the correct way here, is a SID --> QoS mapping. This allows you 
to manage and configure QoS in --one-- place, namely the SM/SA and zero 
effort host side deployment.

For example clustered database has IB subnet both for the inter RAC IPC 
and for the RAC nodes storage access, say IPC is RDS and storage is iSER 
(a similar example from HPC, IPC is MPI and I/O is Lustre)

So you would need to configure an SID to SL mapping at the SA for each 
service and you are --done--. Taking this to cross IB subnets, the SID 
based mapping will also provide the TC and FL for the GRH.

With the above implementation you would have to change each single ULP 
to have it use QoS and you have to have a configuration file also at the 
host side (saying what tos to set through the rdma cm) and not only in 
the SM.

Or.