[ofa-general] Re: [PATCH] opensm: support multiple routers in a subnet

Sasha Khapyorsky sashak at voltaire.com
Mon Nov 19 12:18:44 PST 2007


Hi Rolf,

On 01:08 Fri 16 Nov     , Rolf Manderscheid wrote:
> Hi Sasha,
> 
> If a path record query is made for an off-subnet DGID, the SA needs to
> return a path record where the DLID points to the router port that
> handles the DGID prefix.  In the case of a subnet with only one
> router, the SA could just pick "the router", and that's exactly what
> the ROUTER_EXP code did.  However, ROUTER_EXP did not look beyond the
> first available router.
> 
> When additional routers are added, the SA needs more information.  The
> mechanism for gathering this information has not yet been specified,
> so in the meantime, this patch adds a configuration file that
> specifies which router ports handle which prefixes.
> 
> The patch also removes all occurrences of ROUTER_EXP ifdefs.  The
> default behaviour remains unchanged with one minor exception:
> hop limits are set to 0xFF for path records to multicast DGIDs if
> the scope is non-local and to unicast DGIDs if off-subnet.
> This used to happen for ROUTER_EXP only.
> 
> Now, the same binary can be configured at run-time to enable the
> ROUTER_EXP behaviour with a generic configuration file, or to handle
> multiple routers on a subnet with a more explicit configuration file.
> See the man page for details.
> 
> Signed-off-by: Rolf Manderscheid <rvm at obsidianresearch.com>

Applied. Thanks.

I like the idea and the patch.

However have some mostly minor comments. I will be "out of office" and
likely off-line for next couple of days, so I will do some changes
in-place. The rest (less critical) improvements could be added as
subsequent patch.

> 
> --
> 
> One consequence of this patch is that people accustomed to using
> ROUTER_EXP will need to specify a configuration file to get the
> same behaviour.  I toyed with the idea of keeping one ROUTER_EXP ifdef
> to control the default behaviour, but then we're back to having two
> versions of opensm with different default behaviours, and the
> counter-intuitive: empty cfg file != non-existent cfg file.  One of
> the goals was to get to a single standard binary.  So, to help avoid
> surprises, I actually added back one ifdef ROUTER_EXP which causes the
> compilation to fail with a useful message.  This only helps those who
> both build and configure their special ROUTER_EXP opensms, but I
> suspect that's most.
> 
> Thanks to Hal for reviewing early versions of this patch and providing
> feedback.
> 
>     Rolf
> 
> ---
> diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h
> index aa8d378..db58919 100644
> --- a/opensm/include/opensm/osm_base.h
> +++ b/opensm/include/opensm/osm_base.h
> @@ -253,6 +253,22 @@ BEGIN_C_DECLS
>  #endif /* __WIN__ */
>  /***********/
>  
> +/****d* OpenSM: Base/OSM_DEFAULT_PREFIX_ROUTES_FILE
> +* NAME
> +*	OSM_DEFAULT_PREFIX_ROUTES_FILE
> +*
> +* DESCRIPTION
> +*	Specifies the default prefix routes file name
> +*
> +* SYNOPSIS
> +*/
> +#ifdef __WIN__
> +#define OSM_DEFAULT_PREFIX_ROUTES_FILE strcat(GetOsmCachePath(), "osm-prefix-routes.conf")
> +#else
> +#define OSM_DEFAULT_PREFIX_ROUTES_FILE "/etc/ofa/opensm-prefix-routes.conf"
> +#endif
> +/***********/
> +

Recently we switched to using configurable config directory for OpenSM.
I will add this separately.

>  /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS
>  * NAME
>  *	OSM_DEFAULT_SWEEP_INTERVAL_SECS
> diff --git a/opensm/include/opensm/osm_prefix_route.h b/opensm/include/opensm/osm_prefix_route.h
> new file mode 100644
> index 0000000..cebd532
> --- /dev/null
> +++ b/opensm/include/opensm/osm_prefix_route.h
> @@ -0,0 +1,52 @@
> +/*
> + * Copyright (c) 2004, 2005 Voltaire, Inc. All rights reserved.
> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved.
> + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved.
> + *
> + * This software is available to you under a choice of one of two
> + * licenses.  You may choose to be licensed under the terms of the GNU
> + * General Public License (GPL) Version 2, available from the file
> + * COPYING in the main directory of this source tree, or the
> + * OpenIB.org BSD license below:
> + *
> + *     Redistribution and use in source and binary forms, with or
> + *     without modification, are permitted provided that the following
> + *     conditions are met:
> + *
> + *      - Redistributions of source code must retain the above
> + *        copyright notice, this list of conditions and the following
> + *        disclaimer.
> + *
> + *      - Redistributions in binary form must reproduce the above
> + *        copyright notice, this list of conditions and the following
> + *        disclaimer in the documentation and/or other materials
> + *        provided with the distribution.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
> + * SOFTWARE.
> + *
> + */
> +
> +#ifndef _OSM_PREFIX_ROUTES_H_
> +#define _OSM_PREFIX_ROUTES_H_
> +
> +#include <complib/cl_types_osd.h>

We are not using *_osd.h header files directly. There should be just

	#include <complib/cl_types.h>

Changing this.

> +#include <complib/cl_qlist.h>
> +
> +typedef struct {
> +	cl_list_item_t list_item;	/* must be first */
> +	uint64_t prefix;		/* network order, zero means "any" */
> +	uint64_t guid;			/* network order, zero means "any" */
> +} osm_prefix_route_t;
> +
> +#ifdef ROUTER_EXP
> +#error ROUTER_EXP is deprecated, specify prefix routes at runtime instead (see opensm man page for details)
> +#endif
> +
> +#endif /* _OSM_PREFIX_ROUTES_H_ */

What do you think? Will it be helpful to merge osm_prefix_routes.h and
osm_router.h (or actually to put 'struct osm_prefix_route' definition
in osm_router.h)?

When adding new header file - you need to add this to
include/Makefile.am EXTRA_DIST list, it is in order to not break
'make dist' functionality. I'm adding this now.

> diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h
> index 452098b..b67add3 100644
> --- a/opensm/include/opensm/osm_subnet.h
> +++ b/opensm/include/opensm/osm_subnet.h
> @@ -54,6 +54,7 @@
>  #include <complib/cl_ptr_vector.h>
>  #include <complib/cl_list.h>
>  #include <opensm/osm_base.h>
> +#include <opensm/osm_prefix_route.h>
>  
>  #ifdef __cplusplus
>  #  define BEGIN_C_DECLS extern "C" {
> @@ -298,6 +299,7 @@ typedef struct _osm_subn_opt {
>  #endif				/* ENABLE_OSM_PERF_MGR */
>  	char *event_plugin_name;
>  	char *node_name_map_name;
> +	char *prefix_routes_file;
>  } osm_subn_opt_t;
>  /*
>  * FIELDS
> @@ -550,6 +552,7 @@ typedef struct _osm_subn {
>  	cl_qmap_t node_guid_tbl;
>  	cl_qmap_t port_guid_tbl;
>  	cl_qmap_t rtr_guid_tbl;
> +	cl_qlist_t prefix_routes_list;
>  	cl_qmap_t prtn_pkey_tbl;
>  	cl_qmap_t mgrp_mlid_tbl;
>  	cl_qmap_t sm_guid_tbl;
> diff --git a/opensm/man/opensm.8 b/opensm/man/opensm.8
> index ef12980..d5434be 100644
> --- a/opensm/man/opensm.8
> +++ b/opensm/man/opensm.8
> @@ -20,6 +20,7 @@ opensm \- InfiniBand subnet manager and administration (SM/SA)
>  [\-Q | \-\-qos] [\-N | \-\-no_part_enforce] [\-y | \-\-stay_on_fatal]
>  [\-B | \-\-daemon] [\-I | \-\-inactive]
>  [\-\-perfmgr] [\-\-perfmgr_sweep_time_s <seconds>]
> +[\-\-prefix_routes_file <path>]
>  [\-v(erbose)] [\-V] [\-D <flags>] [\-d(ebug) <number>] [\-h(elp)] [\-?]
>  
>  .SH DESCRIPTION
> @@ -201,6 +202,13 @@ is accumulative.
>  This option defines the optional partition configuration file.
>  The default name is \'/etc/opensm/opensm-partitions.conf\'.
>  .TP
> +.BI --prefix_routes_file= path

I guess here should be also something like:

"This option specifies the prefix routes file..."

> +Prefix routes control how the SA responds to path record queries for
> +off-subnet DGIDs.  By default, the SA fails such queries. The
> +.B PREFIX ROUTES
> +section below describes the format of the configuration file.
> +The default path is \fB\%/etc/ofa/opensm\-prefix\-routes.conf\fP.
> +.TP
>  \fB\-Q\fR, \fB\-\-qos\fR
>  This option enables QoS setup. It is disabled by default.
>  .TP
> @@ -465,6 +473,39 @@ Examples:
>   qos_ca_sl2vl=0,1,2,3,5,5,5,12,12,0,
>   qos_swe_high_limit=0
>  
> +.SH PREFIX ROUTES
> +.PP
> +Prefix routes control how the SA responds to path record queries for
> +off-subnet DGIDs.  By default, the SA fails such queries.
> +Note that IBA does not specify how the SA should obtain off-subnet path
> +record information.
> +The prefix routes configuration is meant as a stop-gap until the
> +specification is completed.
> +.PP
> +Each line in the configuration file is a 64-bit prefix followed by a
> +64-bit GUID, separated by white space.
> +The GUID specifies the router port on the local subnet that will
> +handle the prefix.
> +Blank lines are ignored, as is anything between a \fB#\fP character
> +and the end of the line.
> +The prefix and GUID are both in hex, the leading 0x is optional.
> +Either, or both, can be wild-carded by specifying an
> +asterisk instead of an explicit prefix or GUID.
> +.PP
> +When responding to a path record query for an off-subnet DGID,
> +opensm searches for the first prefix match in the configuration file.
> +Therefore, the order of the lines in the configuration file is important:
> +a wild-carded prefix at the beginning of the configuration file renders
> +all subsequent lines useless.
> +If there is no match, then opensm fails the query.
> +It is legal to repeat prefixes in the configuration file,
> +opensm will return the path to the first available matching router.
> +A configuration file with a single line where both prefix and GUID
> +are wild-carded means that a path record query specifying any
> +off-subnet DGID should return a path to the first available router.
> +This configuration yields the same behaviour formerly achieved by
> +compiling opensm with -DROUTER_EXP.
> +
>  .SH ROUTING
>  .PP
>  OpenSM now offers five routing engines:
> @@ -872,6 +913,10 @@ a file compatible with dump_lfts.sh output. This file can be used
>  as input for forwarding tables loading by 'file' routing engine.
>  Both or one of options -U and -M can be specified together with \'-R file\'.
>  
> +.SH FILES
> +.TP
> +.B /etc/ofa/opensm-prefix-routes.conf
> +default prefix routes file.
>  
>  .SH AUTHORS
>  .TP
> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c
> index 13c9f70..4b99dd0 100644
> --- a/opensm/opensm/main.c
> +++ b/opensm/opensm/main.c
> @@ -291,6 +291,11 @@ void show_usage(void)
>  	printf("--perfmgr_sweep_time_s <sec.>\n"
>  	       "           PerfMgr sweep interval in seconds.\n\n");
>  #endif
> +	printf("--prefix_routes_file <path to file>\n"
> +	       "          This option specifies the prefix routes file.\n"
> +	       "          Prefix routes control how the SA responds to path record\n"
> +	       "          queries for off-subnet DGIDs.  Default file is:\n"
> +	       "              "OSM_DEFAULT_PREFIX_ROUTES_FILE"\n\n");
>  	printf("-v\n"
>  	       "--verbose\n"
>  	       "          This option increases the log verbosity level.\n"
> @@ -609,6 +614,7 @@ int main(int argc, char *argv[])
>  		{"perfmgr", 0, NULL, 1},
>  		{"perfmgr_sweep_time_s", 1, NULL, 2},
>  #endif
> +		{"prefix_routes_file", 1, NULL, 3},
>  		{NULL, 0, NULL, 0}	/* Required at the end of the array */
>  	};
>  
> @@ -911,6 +917,9 @@ int main(int argc, char *argv[])
>  			break;
>  #endif				/* ENABLE_OSM_PERF_MGR */
>  
> +		case 3:
> +			opt.prefix_routes_file = optarg;
> +			break;
>  		case 'h':
>  		case '?':
>  		case ':':
> diff --git a/opensm/opensm/osm_sa_path_record.c b/opensm/opensm/osm_sa_path_record.c
> index ce75ec8..2597046 100644
> --- a/opensm/opensm/osm_sa_path_record.c
> +++ b/opensm/opensm/osm_sa_path_record.c
> @@ -69,10 +69,9 @@
>  #include <opensm/osm_opensm.h>
>  #include <opensm/osm_qos_policy.h>
>  #include <opensm/osm_sa.h>
> -#ifdef ROUTER_EXP
>  #include <opensm/osm_router.h>
>  #include <opensm/osm_sa_mcmember_record.h>
> -#endif
> +#include <opensm/osm_prefix_route.h>
>  
>  #define OSM_PR_RCV_POOL_MIN_SIZE    64
>  #define OSM_PR_RCV_POOL_GROW_SIZE   64
> @@ -858,19 +857,12 @@ __osm_pr_rcv_build_pr(IN osm_pr_rcv_t * const p_rcv,
>  {
>  	const osm_physp_t *p_src_physp;
>  	const osm_physp_t *p_dest_physp;
> -#ifdef ROUTER_EXP
>  	boolean_t is_nonzero_gid = 0;
> -#endif
>  
>  	OSM_LOG_ENTER(p_rcv->p_log, __osm_pr_rcv_build_pr);
>  
>  	p_src_physp = p_src_port->p_physp;
> -#ifndef ROUTER_EXP
> -	p_dest_physp = p_dest_port->p_physp;
>  
> -	p_pr->dgid.unicast.prefix = osm_physp_get_subnet_prefix(p_dest_physp);
> -	p_pr->dgid.unicast.interface_id = osm_physp_get_port_guid(p_dest_physp);
> -#else
>  	if (p_dgid) {
>  		if (memcmp(p_dgid, &zero_gid, sizeof(*p_dgid)))
>  			is_nonzero_gid = 1;
> @@ -886,7 +878,6 @@ __osm_pr_rcv_build_pr(IN osm_pr_rcv_t * const p_rcv,
>  		p_pr->dgid.unicast.interface_id =
>  		    osm_physp_get_port_guid(p_dest_physp);
>  	}
> -#endif
>  
>  	p_pr->sgid.unicast.prefix = osm_physp_get_subnet_prefix(p_src_physp);
>  	p_pr->sgid.unicast.interface_id = osm_physp_get_port_guid(p_src_physp);
> @@ -895,11 +886,10 @@ __osm_pr_rcv_build_pr(IN osm_pr_rcv_t * const p_rcv,
>  	p_pr->slid = cl_hton16(src_lid_ho);
>  
>  	p_pr->hop_flow_raw &= cl_hton32(1 << 31);
> -#ifdef ROUTER_EXP
> +
>  	/* Only set HopLimit if going through a router */
>  	if (is_nonzero_gid)
>  		p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
> -#endif
>  
>  	p_pr->pkey = p_parms->pkey;
>  	ib_path_rec_set_sl(p_pr, p_parms->sl);
> @@ -1262,10 +1252,8 @@ __osm_pr_rcv_get_end_points(IN osm_pr_rcv_t * const p_rcv,
>  	ib_net64_t dest_guid;
>  	ib_api_status_t status;
>  	ib_net16_t sa_status = IB_SA_MAD_STATUS_SUCCESS;
> -#ifdef ROUTER_EXP
>  	osm_router_t *p_rtr;
>  	osm_port_t *p_rtr_port;
> -#endif
>  
>  	OSM_LOG_ENTER(p_rcv->p_log, __osm_pr_rcv_get_end_points);
>  
> @@ -1359,20 +1347,47 @@ __osm_pr_rcv_get_end_points(IN osm_pr_rcv_t * const p_rcv,
>  					"Non local DGID subnet prefix 0x%016"
>  					PRIx64 "\n",
>  					cl_ntoh64(p_pr->dgid.unicast.prefix));

This is part of osm_log(.. OSM_LOG_VERBOSE ..), we are going to make
this prefix routes support to be mainstream, do you think verbosity of
this message should be decreased (to let's say OSM_LOG_DEBUG)?

> -#ifndef ROUTER_EXP
> -				/*
> -				   This 'error' is the client's fault (bad gid) so
> -				   don't enter it as an error in our own log.
> -				   Return an error response to the client.
> -				 */
> -				sa_status = IB_SA_MAD_STATUS_INVALID_GID;
> -				goto Exit;
> -#else
> -				/* Just use "first" router (if it exists) for now */
> -				p_rtr =
> -				    (osm_router_t *) cl_qmap_head(&p_rcv->
> -								  p_subn->
> -								  rtr_guid_tbl);
> +
> +				/* Find the router port that is configured to handle
> +				   this prefix, if any: */
> +				osm_prefix_route_t *route = NULL;
> +				osm_prefix_route_t *r = (osm_prefix_route_t *)
> +					cl_qlist_head(&p_rcv->p_subn->prefix_routes_list);
> +
> +				while (r != (osm_prefix_route_t *)
> +				       cl_qlist_end(&p_rcv->p_subn->prefix_routes_list))
> +				{
> +					if (r->prefix == p_pr->dgid.unicast.prefix ||
> +					    r->prefix == 0)
> +					{
> +						route = r;
> +						break;
> +					}
> +					r = (osm_prefix_route_t *) cl_qlist_next(&r->list_item);
> +				}
> +
> +				if (! route) {
> +					/*
> +					  This 'error' is the client's fault (bad gid) so
> +					  don't enter it as an error in our own log.
> +					  Return an error response to the client.
> +					*/
> +					sa_status = IB_SA_MAD_STATUS_INVALID_GID;
> +					goto Exit;
> +				} else if (route->guid == 0) {
> +					/* first router */
> +					p_rtr = (osm_router_t *)
> +						cl_qmap_head(&p_rcv->
> +							     p_subn->
> +							     rtr_guid_tbl);
> +				} else {
> +					p_rtr = (osm_router_t *)
> +						cl_qmap_get(&p_rcv->
> +							    p_subn->
> +							    rtr_guid_tbl,
> +							    route->guid);
> +				}
> +
>  				if (p_rtr ==
>  				    (osm_router_t *) cl_qmap_end(&p_rcv->
>  								 p_subn->

All this "non-local" section looks for me as good candidate for separate
function.

> @@ -1380,7 +1395,7 @@ __osm_pr_rcv_get_end_points(IN osm_pr_rcv_t * const p_rcv,
>  				{
>  					osm_log(p_rcv->p_log, OSM_LOG_ERROR,
>  						"__osm_pr_rcv_get_end_points: ERR 1F22: "
> -						"Off subnet DGID but no routers found\n");
> +						"Off subnet DGID but router not found\n");
>  					sa_status =
>  					    IB_SA_MAD_STATUS_INVALID_GID;
>  					goto Exit;
> @@ -1390,7 +1405,6 @@ __osm_pr_rcv_get_end_points(IN osm_pr_rcv_t * const p_rcv,
>  				dest_guid = osm_port_get_guid(p_rtr_port);
>  				if (p_dgid)
>  					*p_dgid = p_pr->dgid;
> -#endif
>  			}
>  		}
>  
> @@ -2134,22 +2148,14 @@ void osm_pr_rcv_process(IN void *context, IN void *data)
>  					  &sl, &flow_label, &hop_limit);
>  		ib_path_rec_set_sl(&p_pr_item->path_rec, sl);
>  		ib_path_rec_set_qos_class(&p_pr_item->path_rec, 0);
> -#ifndef ROUTER_EXP
> -		p_pr_item->path_rec.hop_flow_raw =
> -		    cl_hton32(hop_limit) | (flow_label << 8);
> -#else
> +
>  		/* HopLimit is not yet set in non link local MC groups */
>  		/* If it were, this would not be needed */
> -		if (ib_mgid_get_scope
> -		    (&p_mgrp->mcmember_rec.mgid) == MC_SCOPE_LINK_LOCAL)
> -			p_pr_item->path_rec.
> -			    hop_flow_raw =
> -			    cl_hton32(hop_limit) | (flow_label << 8);
> -		else
> -			p_pr_item->path_rec.
> -			    hop_flow_raw =
> -			    cl_hton32(IB_HOPLIMIT_MAX) | (flow_label << 8);
> -#endif
> +		if (ib_mgid_get_scope(&p_mgrp->mcmember_rec.mgid) != MC_SCOPE_LINK_LOCAL)
> +			hop_limit = IB_HOPLIMIT_MAX;
> +
> +		p_pr_item->path_rec.hop_flow_raw =
> +			cl_hton32(hop_limit) | (flow_label << 8);
>  
>  		cl_qlist_insert_tail(&pr_list, (cl_list_item_t *)
>  				     & p_pr_item->pool_item);
> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c
> index 0f109a5..834d283 100644
> --- a/opensm/opensm/osm_subnet.c
> +++ b/opensm/opensm/osm_subnet.c
> @@ -91,6 +91,7 @@ void osm_subn_construct(IN osm_subn_t * const p_subn)
>  	cl_qmap_init(&p_subn->sm_guid_tbl);
>  	cl_qlist_init(&p_subn->sa_sr_list);
>  	cl_qlist_init(&p_subn->sa_infr_list);
> +	cl_qlist_init(&p_subn->prefix_routes_list);

And also prefix_routes_list elements should be freed on subnet object
destruction. I'm adding this now.

Thanks for your great work!
Sasha

>  	cl_qmap_init(&p_subn->rtr_guid_tbl);
>  	cl_qmap_init(&p_subn->prtn_pkey_tbl);
>  	cl_qmap_init(&p_subn->mgrp_mlid_tbl);
> @@ -475,6 +476,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt)
>  	p_opt->exit_on_fatal = TRUE;
>  	p_opt->enable_quirks = FALSE;
>  	p_opt->no_clients_rereg = FALSE;
> +	p_opt->prefix_routes_file = OSM_DEFAULT_PREFIX_ROUTES_FILE;
>  	subn_set_default_qos_options(&p_opt->qos_options);
>  	subn_set_default_qos_options(&p_opt->qos_ca_options);
>  	subn_set_default_qos_options(&p_opt->qos_sw0_options);
> @@ -686,6 +688,112 @@ subn_dump_qos_options(FILE * file,
>  
>  /**********************************************************************
>   **********************************************************************/
> +static ib_api_status_t
> +append_prefix_route(IN osm_subn_t * const p_subn, uint64_t prefix, uint64_t guid)
> +{
> +	osm_prefix_route_t *route;
> +
> +	route = malloc(sizeof *route);
> +	if (! route) {
> +		osm_log(&p_subn->p_osm->log, OSM_LOG_ERROR, "%s: out of memory", __FUNCTION__);
> +		return IB_ERROR;
> +	}
> +
> +	route->prefix = cl_hton64(prefix);
> +	route->guid = cl_hton64(guid);
> +	cl_qlist_insert_tail(&p_subn->prefix_routes_list, &route->list_item);
> +	return IB_SUCCESS;
> +}
> +
> +static ib_api_status_t
> +osm_parse_prefix_routes_file(IN osm_subn_t * const p_subn)
> +{
> +	osm_log_t *log = &p_subn->p_osm->log;
> +	FILE *fp;
> +	char buf[1024];
> +	int line = 0;
> +	int errors = 0;
> +
> +	while (! cl_is_qlist_empty(&p_subn->prefix_routes_list)) {
> +		cl_list_item_t *item = cl_qlist_remove_head(&p_subn->prefix_routes_list);
> +		free(item);
> +	}
> +
> +	fp = fopen(p_subn->opt.prefix_routes_file, "r");
> +	if (! fp) {
> +		if (errno == ENOENT)
> +			return IB_SUCCESS;
> +
> +		osm_log(log, OSM_LOG_ERROR, "%s: fopen(%s) failed: %s",
> +			__FUNCTION__, p_subn->opt.prefix_routes_file, strerror(errno));
> +		return IB_ERROR;
> +	}
> +
> +	while (fgets(buf, sizeof buf, fp) != NULL) {
> +		char *p_prefix, *p_guid, *p_extra, *p_last, *p_end;
> +		uint64_t prefix, guid;
> +
> +		line++;
> +		if (errors > 10)
> +			break;
> +
> +		p_prefix = strtok_r(buf, " \t\n", &p_last);
> +		if (! p_prefix)
> +			continue; /* ignore blank lines */
> +
> +		if (*p_prefix == '#')
> +			continue; /* ignore comment lines */
> +
> +		p_guid = strtok_r(NULL, " \t\n", &p_last);
> +		if (! p_guid) {
> +			osm_log(log, OSM_LOG_ERROR, "%s:%d: missing GUID\n",
> +				p_subn->opt.prefix_routes_file, line);
> +			errors++;
> +			continue;
> +		}
> +
> +		p_extra = strtok_r(NULL, " \t\n", &p_last);
> +		if (p_extra && *p_extra != '#') {
> +			osm_log(log, OSM_LOG_INFO, "%s:%d: extra tokens ignored\n",
> +				p_subn->opt.prefix_routes_file, line);
> +		}
> +
> +		if (strcmp(p_prefix, "*") == 0)
> +			prefix = 0;
> +		else {
> +			prefix = strtoull(p_prefix, &p_end, 16);
> +			if (*p_end != '\0') {
> +				osm_log(log, OSM_LOG_ERROR, "%s:%d: illegal prefix: %s\n",
> +					p_subn->opt.prefix_routes_file, line, p_prefix);
> +				errors++;
> +				continue;
> +			}
> +		}
> +
> +		if (strcmp(p_guid, "*") == 0)
> +			guid = 0;
> +		else {
> +			guid = strtoull(p_guid, &p_end, 16);
> +			if (*p_end != '\0' && *p_end != '#') {
> +				osm_log(log, OSM_LOG_ERROR, "%s:%d: illegal GUID: %s\n",
> +					p_subn->opt.prefix_routes_file, line, p_guid);
> +				errors++;
> +				continue;
> +			}
> +		}
> +
> +		if (append_prefix_route(p_subn, prefix, guid) != IB_SUCCESS) {
> +			errors++;
> +			break;
> +		}
> +	}
> +
> +	fclose(fp);
> +	return (errors == 0) ? IB_SUCCESS : IB_ERROR;
> +}
> +
> +/**********************************************************************
> + **********************************************************************/
>  ib_api_status_t osm_subn_rescan_conf_files(IN osm_subn_t * const p_subn)
>  {
>  	char *p_cache_dir = getenv("OSM_CACHE_DIR");
> @@ -745,6 +853,8 @@ ib_api_status_t osm_subn_rescan_conf_files(IN osm_subn_t * const p_subn)
>  	if (p_subn->opt.qos)
>  		osm_qos_parse_policy_file(p_subn);
>  
> +	osm_parse_prefix_routes_file(p_subn);
> +
>  	return IB_SUCCESS;
>  }
>  
> @@ -1285,6 +1395,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts)
>  
>  		opts_unpack_boolean("no_clients_rereg",
>  				    p_key, p_val, &p_opts->no_clients_rereg);
> +
> +		opts_unpack_charp("prefix_routes_file",
> +				  p_key, p_val, &p_opts->prefix_routes_file);
>  	}
>  	fclose(opts_file);
>  
> @@ -1606,6 +1719,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts)
>  			      "QoS Router ports options", "qos_rtr",
>  			      &p_opts->qos_rtr_options);
>  
> +	fprintf(opts_file,
> +		"# Prefix routes file name\n"
> +		"prefix_routes_file %s\n\n",
> +		p_opts->prefix_routes_file);
> +
>  	/* optional string attributes ... */
>  
>  	fclose(opts_file);



More information about the general mailing list