[ofa-general] [PATCH v2] OFED docs/opensm_release_notes.txt: OpenSM 3.2.5 RN
Yevgeny Kliteynik
kliteyn at dev.mellanox.co.il
Tue Dec 9 01:47:02 PST 2008
OpenSM 3.2.5 Release Notes for OFED 1.4 docs.
V2: add "opensm -Q" error messages as known issues.
Signed-off-by: Yevgeny Kliteynik <kliteyn at dev.mellanox.co.il>
---
opensm_release_notes.txt | 406 ++++++++++++++++++++++++++++++----------------
1 files changed, 268 insertions(+), 138 deletions(-)
diff --git a/opensm_release_notes.txt b/opensm_release_notes.txt
index 007c85a..11223de 100644
--- a/opensm_release_notes.txt
+++ b/opensm_release_notes.txt
@@ -1,121 +1,209 @@
- OpenSM Release Notes 3.1.11
+ OpenSM Release Notes 3.2
=============================
-Version: OpenFabrics Enterprise Distribution (OFED) 1.3
-Repo: git://git.openfabrics.org/~ofed_1_3/management.git (release)
- git://git.openfabrics.org/~sashak/management.git (development)
-Date: June 2008
+Version: OpenSM 3.2.x
+Repo: git://git.openfabrics.org/~sashak/management.git
+Date: Dec 2008
1 Overview
----------
-This document describes the contents of the OpenSM OFED 1.3 release.
+This document describes the contents of the OpenSM 3.2 release.
OpenSM is an InfiniBand compliant Subnet Manager and Administration,
and runs on top of OpenIB. The OpenSM version for this release
-is openib-3.1.11
+is opensm-3.2.5
This document includes the following sections:
1 This Overview section (describing new features and software
dependencies)
2 Known Issues And Limitations
3 Unsupported IB compliance statements
-4 Major Bug Fixes
+4 Bug Fixes
5 Main Verification Flows
-6 Qualified software stacks and devices
+6 Qualified Software Stacks and Devices
1.1 Major New Features
-* QoS manager (experimental)
- This QoS manager implementation is in accordance with IBA QoS Annex.
- Highly configurable QoS Policy is parsed from OpenSM QoS policy file.
- Valid QoS parameters will be reported in SA PathRecord and
- MultiPathRecord. In addition simple QoS levels per ULPs configuration
- is supported too.
-
-* Performance Manager
- When enabled it collects a fabric port counters and able to log it or
- to pass to external program via event plugin interface. It handles
- counters overflow, supports LID/QP redirection and is able to work
- when OpenSM is in master, standby, and inactive states.
-
-* Dimension Order routing (DOR) algorithm
- DOR Unicast routing algorithm - based on the Min Hop algorithm, but
- avoids port equalization except for redundant links between the
- same two switches. This provides deadlock free routes for hypercubes
- when the fabric is cabled as a hypercube and for meshes when cabled
- as a mesh (see details in OpenSM man page).
-
-* Routing improvements
- Speedup the current routing algorithms default MinHops, Up/Down and
- LASH and lid matrix generation. Fat Tree routing engine is able to work
- with not pure fat free topology.
-
-* Multiple IB routers support
- OpenSM now able to keep configurable subnet prefix to router table.
- SA will report path to this routers when SA PathRecord was issued with
- non-local DGID.
-
-* Node map
- This is possible to name nodes in this config file. Those names will be
- used for logging and by QoS configuration.
-
-* PKey index support
- Proper support for PKey index in GSI queries.
-
-* Incremental LFTs, PKey, SL2VL, and VLarbitration table updates
- OpenSM will only fetch those tables in first heavy sweep and then
- will maintain this internally.
-
-* Fast port and switch detector
- When port and/or switch was externally reset and it was fast so sweep
- doesn't find this device as disconnected OpenSM will detect this by
- changed port states and handle accordingly.
-
-* Duplicated GUIDs/port moving detector
- OpenSM will be able to detect port moving during a fabric discovery
- and will not report duplicated GUIDs in this case.
-
-* Multicast rerouting speedup
- Now OpenSM will calculate and setup multicast forwarding tables for
- all altered multicast groups and not for each one.
-
-* Event plugin API
- OpenSM allows to load dynamically various plugin modules.
-
-* Many generic improvements
+* Cached Routing
+ OpenSM provides an optional unicast routing cache (enabled by '-A' or
+ '--ucast_cache' options). When enabled, unicast routing cache prevents
+ routing recalculation (which is a heavy task in a large cluster) when
+ there was no topology change detected during the heavy sweep, or when
+ the topology change does not require new routing calculation, e.g. when
+ one or more CAs/RTRs/leaf switches going down, or one or more of these
+ nodes coming back after being down.
+
+* Routing Chaining
+ Routing chaining is the ability to configure the order in which routing
+ algorithms are applied in opensm, i.e. '-R ftree,updn,minhop' - try
+ using ftree routing. If ftree fails, try updn. If updn fails, try
+ minhop.
+
+* IPv6 Solicited Node Multicast addresses consolidation
+ When this mode is used (enabled with --consolidate_ipv6_snm_req option)
+ OpenSM will map all IPv6 Solicited Node Multicast address join requests
+ into a single Multicast group with address ff10:601b::1:ff00:0. In this
+ way limited MLID space is saved. This IBA noncompliant feature is very
+ useful with large (~> 1024 nodes) clusters.
+
+* OpenSM sweep state machine rework
+ Huge and buggy OpenSM sweep state machine was fully rewritten in safer
+ and more effective synchronous manner.
+
+* Multi lid routing balancing for updn/minhop routing algorithms
+ When LMC > 0 is used OpenSM will ensure to generate routing paths via
+ different switches and when possible chassis.
+
+* Preserve base lid routes when LMC > 0
+ When LMC > 0 is used OpenSM will preserve routing paths for base lids
+ as it would be with LMC = 0. In this way traffic on each LID level is
+ not affected by LMC changes.
+
+* Ordered routing paths balancing
+ This adds ability to predefine the port order in which routing paths
+ balancing is performed by OpenSM. Helps to improve performance
+ dramatically (40-50%) for applications with known communication
+ pattern. Activated with --guid_routing_order_file command line option.
+
+* Unified OpenSM configuration
+ Now there is "conventional" config file instead of hidden option cache
+ file (opensm.opts). OpenSM will find this in a default place (consult
+ man page for exact value) or the file name can be specified with '-F'
+ command line option. Also there is an option ('-c') to generate config
+ file template.
+
+* Query remote SMs during light sweep
+ Master OpenSM will query remote standby SMs periodically to catch its
+ possible state changes and react accordingly (as required by IBA spec).
+
+* Predefined port ids for Up/Down algorithm
+ This is useful as Up/Down fine tuning tool - the algorithm will use
+ predefined port IDs instead of GUIDs for its decision about direction.
+ Activated with --ids_guid_file command line option.
+
+* Improved plugin API version 2.
+ Now OpenSM will provide to plugins the access to all data structures.
+ This make it possible to implement powerful multi purpose plugins. All
+ OpenSM header files are installed now and specific configuration/build
+ options are exported via generated osm_config.h header file.
+
+* Many code improvements, optimizations and cleanups
+
+* Automatic daily snapshots generation.
+ This is is not a "feature", but simplifies the access to recent OpenSM
+ bits.
1.2 Minor New Features:
-* Daemon mode can be activated with -B option.
+* Cleanup cl_qlock_pool memory allocator - speedup memory allocations
-* Support multiple scopes for IPoIB multicast groups in partition config.
+* Support for configurable (via OSM_UMAD_MAX_PENDING environment variable)
+ size of pending MADs pool.
-* Loopback connection handling
- Loopback connection is not interpreted as duplicated GUID anymore.
+* Set packet life time to subnet timeout option rather than default
-* Connect root nodes option for Up/Down routing engine.
- When this option is specified Up/Down will create routing paths between
- its root nodes.
+* Enforce routing paths rebalancing on switch reconnection
-* Dump and log filenames changed from osm* to opensm*.
+* In Up/Down routing algorithm compare GUID values in host byte order
-* Support loopback console
- Socket console with only local access.
+* Add 'switchbalance' and 'lidbalance' commands for OpenSM console
-* Configurable config directory (the default value is /etc/opensm) and
- configurable default values of OpenSM config filenames.
+* Respond to new trap 144 node description update flag
-* Add option for force SDR link speed
- Add option to opensm.opts to force link speed. Currently, only forcing
- to SDR link speed is supported. This option is not supported as a
- command line option.
+* Add '--connect_roots' command line options. This preserves connectivity
+ between root nodes in Up/Down routing algorithm
-* Better packaging
- Building and RPM packaging were improved and simplified.
+* Setting SL in the IPoIB MCast groups in accordance with QoS policy
-* Handle "babbling" ports
- When a babbling port (port which causes a frequent trap generation) is
- detected, OpenSM will disable the port which should terminate the trap
- storm.
+* Dump auto detected root node guids in Up/Down routing algorithm
+
+* Unify OpenSM dumpers code
+
+* Unify various guid files parsers - add generic nodenamemap style parser
+
+* When root node guids were provided in file update the list on each
+ Up/Down run
+
+* During ./configure show values of configuration dirs and files
+
+* Make prefix routes config file name configurable
+
+* Add a Performance Manager HOWTO to the docs and the dist
+
+* Support separate SA and SM keys as clarified in IBA 1.2.1
+
+* Remove AM_MAINTAINER_MODE in ./configure
+
+* Make vendor type OSM_VENDOR_INTF_OPENIB (libibumad) to be default
+
+* Build osm_perfmgr_db.* content only when PerfMgr is enabled.
+
+* Move PerfMgr event_db_dump_file to common OpenSM dump dir
+
+* Allow space separated strings as values in OpenSM config
+
+* Support for multiple event plugins
+
+* Add '--version' command line option
+
+* Add '--create-config <file-name>' command line option
+
+* Speedup and simplify logging code
+
+* Speedup multicast processing in SA DB
+
+* In log messages convert unicast LIDs from hex to decimal format and
+ GIDs from hex to IPv6 address format
+
+* Handle all possible ports in "ignore-guids" file
+
+* Add 'reroute' console command
+
+* Remove many install-exec-hook from Makefiles
+
+* Some cleanups in LASH routing algorithm code
+
+* In Makefiles remove -rpath and explicit -lpthread, -ldl from LDFLAGS
+ (move to configurator)
+
+* Install all OpenSM header files
+
+* Improve locking in SM Info receiver
+
+* Add new OSM_EVENT_ID_SUBNET_UP event for plugins
+
+* Redo lex and yacc files generation in conventional way
+
+* Add a missing Node Description check on light sweep.
+
+* Move vendor specific compilation defines from command to generated
+ config.h file
+
+* Provide useful error message when log file opening fails
+
+* Add generated osm_config.h file with OpenSM specific defines
+
+* Display port number in decimal in log messages
+
+* Replace osm_vendor_select.h by generated osm_config.h
+
+* Unify options listing in OpenSM usage message
+
+* LFT buffers handling simplification
+
+* Add 'dump_conf' console command
+
+* OpenSM performs sweep on SIGCONT (coming out of suspend).
+
+* When our SM is in Standby state and its priority is increased
+ (via console command), notify master SM by sending Trap 144.
+
+* When entering standby state (after discovery) notify master SM
+ with Trap 144.
+
+* support more PortInfo:CapabilityMask bits
+
+* When babbling port policy is on disable the port with the least hop
+ count.
1.3 Library API Changes
@@ -123,12 +211,12 @@ This document includes the following sections:
1.4 Software Dependencies
-OpenSM depends on the installation of either OFED 1.3, OFED 1.2, OFED 1.1,
-OFED 1.0, OpenIB gen2 (e.g. IBG2 distribution), OpenIB gen1 (e.g. IBGD
-distribution), or Mellanox VAPI stacks. The qualified driver versions
-are provided in Table 2, "Qualified IB Stacks".
+OpenSM depends on the installation of either OFED 1.x, OpenIB gen2 (e.g.
+IBG2 distribution), OpenIB gen1 (e.g. IBGD distribution), or Mellanox
+VAPI stacks. The qualified driver versions are provided in Table 2,
+"Qualified IB Stacks".
-Also building of QoS manager policy file parser requires flex, and either
+Also, building of QoS manager policy file parser requires flex, and either
bison or byacc installed.
1.5 Supported Devices Firmware
@@ -147,6 +235,10 @@ are listed in Table 3.
Puts the burden of re-registering services, multicast groups, and
inform-info on the client application (or IB access layer core).
+* When running with QoS with default configuration (opensm -Q),
+ OpenSM prints list of "Invalid Cached Option" error messages.
+ This does not affect OpenSM functionality.
+
3 Unsupported IB Compliance Statements
--------------------------------------
The following section lists all the IB compliance statements which
@@ -210,76 +302,105 @@ information regarding each compliance statement.
* C15-0.1.14 (Services):
Provide means to associate service name and ServiceKeys.
-4 Major Bug Fixes
------------------
+4 Bug Fixes
+-----------
+
+4.1 Major Bug Fixes
-The following is a list of bugs that were fixed. Note that other less critical
-or visible bugs were also fixed.
+* Set SA attribute offset to 0 when no records are returned
-* osm_ucast_ftree.c: do load-leveling of non-CN routes
+* Send trap 64 only after new ports are in ACTIVE state.
-* osm_ucast_ftree.c: ignore port 0 and loopbacks on switches
+* Fix in sending client reregistration bit
-* lash: fix possible segfault in osm_get_lash_sl()
+* Fix default OpenSM SM (and SA) Key byte order
-* osm_ucast_ftree.c: fixing coredump in fat-tree routing
+* Fix in sending Multicast groups creation/deletion notification (Traps
+ 66,67)
-* osm_sa_slvl_record: fix overflow crash
+* Don't startup automatically on SuSE based systems
-* Break multicast rerouting requests processing when heavy sweep is
- scheduled.
+4.2 Other Bug Fixes
-* updn: report fallback properly
+* opensm/osm_console.c: fix seg fault when running "portstatus ca" in
+ the console
-* Fix incorrect identification of routing engine used
+* opensm: fix potential core dumps where osm_node_get_physp_ptr can
+ return NULL
-* Don't zero base LID when invalid value is received
+* opensm/osm_mcast_mgr: limit spanning tree creation recursion to value
+ of max hops (64)
-* lash: fix wrong allocation size
+* opensm: switch LFTs incremental update fix
-* Fixing broken logic in 'process world' part of LinkRecord processing
+* opensm/osm_state_mgr.c: fix segmentation fault
-* Fix lmc_mask bit order in osm_sa_link_record.c
+* opensm: eliminate some potential NULL pointer dereferences
-* Adding missing comparison by to_lid/from_lid in LinkRecord processing
+* opensm/osm_console.c: fix guid parsing
-* Broken logic when scanning subnet for PIR request
+* opensm: fix off by 1 issue with max_lid and max_multicat_lid_ho
-* No interactive games in daemon mode
+* opensm: fix potentially wrong port_guid initialization
-* Fixing memory leak in node description
+* opensm/configure.in: fix wrong HAVE_DEFAULT_OPENSM_CONFIG_FILE define
+ generation
-* Fix PortInfo update issues for switch port 0
+* opensm: fix snprintf() usage
-* Changed method_mask type in user_mad interface in accordance with
- kernel ABI
+* opensm/osm_sa_lft_record: validate LFT block number
-* Use umad_get_issm_path() in osm_vendor_set_sm()
+* opensm/osm_sa_lft_record: pass block parameter in host byte order
-* Report message fix
+* opensm/include/Makefile.am: don't duplicate header files in EXTRA_DIST
-* Uninitialized variables usage fix
+* opensm/osm_sa_class_port_info.c: fix over bound array access
-* osm_ucast_ftree.c: Possible NULL ptr seg fault
+* osmtest/osmt_service.c: fix over bound array access
-* osm_mcast_mgr.c: Possible NULL ptr seg fault
+* osmtest: fix qpn encoding in osmtest_informinfo_request()
-* TrapRepress was failing for mkey != 0
+* opensm/osm_vendor_mlx_sa.c: handling attribute offset of 0
-* IB_PR_COMPMASK was used in MPR
+* opensm: fix segfault corner case when osm_console_init fails
-* Set hop limit when creating ipoib multicast groups
+* opensm/console: close console socket on cleanup path
-* Fix outstanding mad counters tracking on the error paths.
+* opensm/osm_ucast_lash: fix buffer overflow
-* Report new ports before handover mastership
+* opensm: fix broken IPv6 SNM consolidation code
-* Fix opvls and neighbormtu when remote port invalid.
+* opensm/osm_sa_lft_record.c: fix block number encoding byte order
-* Bug in coding trying to set vl_arb_high_limit when PortInfo.base_lid
- was still zero.
+* opensm/osm_sa: fix memory leak in SA responder
-* Protect SMInfo response against port moving issue.
+* opensm/osm_mcast_mgr: fix memory leak
+
+* opensm: fix qos config parsing bugs
+
+* opensm/osm_mcast_tbl.c: fix sending invalid MF block due to max mlid
+ overflow
+
+* opensm: log_max_size config parameter in MB
+
+* opensm/osm_ucast_lash: fix extra memory allocations
+
+* opensm: fix race in main OpenSM flow
+
+* opensm/ftree: fix GUID check against cn_guid_file
+
+* opensm/ftree: save FLT buffers memory allocations
+
+* opensm/osm_sa_link_record.c: prevent potential endless recursion
+
+* opensm: remove SM from sm_guid_tbl when IsSM port capability flag is
+ not set
+
+* opensm: fix QoS config bug
+
+* opensm: don't reassign zeroed params from config file
+
+* Other less critical or visible bugs were also fixed.
5 Main Verification Flows
-------------------------
@@ -439,14 +560,24 @@ interface. The test procedure includes:
* Trap injection and recovery
-6 Qualification
-----------------
+6 Qualified Software Stacks and Devices
+---------------------------------------
+
+OpenSM Compatibility
+--------------------
+Note that OpenSM version 3.2.1 and earlier used a value of 1 in host
+byte order for the default SM_Key, so there is a compatibility issue
+with these earlier versions of OpenSM when the 3.2.2 or later version
+is running on a little endian machine. This affects SM handover as well
+as SA queries (saquery tool in infiniband-diags).
+
Table 2 - Qualified IB Stacks
=============================
Stack | Version
-----------------------------------------|--------------------------
+OFED | 1.4
OFED | 1.3
OFED | 1.2
OFED | 1.1
@@ -463,6 +594,7 @@ Device | FW versions
------------------------------------|-------------------------------
InfiniScale | fw-43132 5.2.000 (and later)
InfiniScale III | fw-47396 0.5.000 (and later)
+InfiniScale IV | fw-48436 7.1.000 (and later)
InfiniHost | fw-23108 3.5.000 (and later)
InfiniHost III Lx | fw-25204 1.2.000 (and later)
InfiniHost III Ex (InfiniHost Mode) | fw-25208 4.8.200 (and later)
@@ -483,10 +615,8 @@ QP0 and QP1. However, it does support it as a device on the subnet.
Note 2: QoS firmware and Mellanox devices
-HCAs: QoS supported by ConnectX. The current FW release
-doesn't support QoS. QoS-enabled FW release (2_5_000) is
-planned for May. If someone wishes to get QoS-enabled FW
-before the official release, they should contact Mellanox FAE.
+HCAs: QoS supported by ConnectX. QoS-enabled FW release is 2_5_000 and
+later.
Switches: QoS supported by InfiniScale III
Any InfiniScale III FW that is supported by OpenSM supports QoS.
--
1.5.1.4
More information about the general
mailing list