[ewg] [PATCH] DAPL: updated release notes for OFED-1.5.3
Davis, Arlin R
arlin.r.davis at intel.com
Mon Feb 28 17:40:54 PST 2011
DAPL release notes for OFED 1.5.3
Signed-off-by: Arlin Davis <arlin.r.davis at intel.com>
---
release_notes/uDAPL_release_notes.txt | 132 ++++++++++++++++++++++++++++++---
1 files changed, 121 insertions(+), 11 deletions(-)
diff --git a/release_notes/uDAPL_release_notes.txt b/release_notes/uDAPL_release_notes.txt
index 60b83bb..d8d68e0 100644
--- a/release_notes/uDAPL_release_notes.txt
+++ b/release_notes/uDAPL_release_notes.txt
@@ -1,24 +1,20 @@
Release Notes for
- OFED 1.5.1 DAPL Release
- March 2010
+ OFED 1.5.3 DAPL Release
+ March 2011
This release of the uDAPL reference implementation package for both
DAT 1.2 and 2.0 specification is timed to coincide with OFED release
of the Open Fabrics (www.openfabrics.org) software stack.
- uDAPL v1 (1.2.16-1) and v2 (2.0.27-1)
+ uDAPL v1 (1.2.19-1) and v2 (2.0.32-1)
----------------
-
- * New Features (v2 only) - UCM provider with IB UD based CM per process.
- More scalable then rdma_cm (cma) or socket cm (scm).
- ----------------
* Provider descriptions and PROS/CONS (cma, scm, ucm)
1. CMA - uses OFA rdma_cm to setup QP's. IPoIB, ARP, and SA queries required.
- Provider name: ofa-v2-cma
+ Provider name: ofa-v2-ib0
PROs: OFA rdma_cm has the most testing across many applications.
Supports both iWARP and IB.
@@ -66,9 +62,11 @@
Settings for larger clusters (512+ cores):
- setenv DAPL_UCM_REP_TIME 800 /* REQUEST timer, waiting for REPLY in millisecs */
- setenv DAPL_UCM_RTU_TIME 400 /* REPLY timer, waiting for RTU in millisecs */
- setenv DAPL_UCM_RETRY 15 /* REQUEST and REPLY retries */
+ setenv DAPL_UCM_REP_TIME 2000 /* REQUEST timer, waiting for REPLY in millisecs */
+ setenv DAPL_UCM_RTU_TIME 2000 /* REPLY timer, waiting for RTU in millisecs */
+ setenv DAPL_UCM_CQ_SIZE 2000 /* CM completion queue */
+ setenv DAPL_UCM_QP_SIZE 2000 /* CM message queue */
+ setenv DAPL_UCM_RETRY 7 /* REQUEST and REPLY retries */
setenv DAPL_ACK_RETRY 7 /* IB RC Ack retry count */
setenv DAPL_ACK_TIMER 20 /* IB RC Ack retry timer */
@@ -114,6 +112,118 @@
V2.0 Package
+ Release 2.0.32 fixes (OFED 1.5.3 GA):
+
+ cma: reduce output log level in disconnect from WARN to CM_WARN
+ ucm: delay freeing of active side UD cm object in case RTU is dropped
+ ucm: cm object needs to be on work queue before req sent on wire
+ ucm,scm: remove use of usec_sleep delays and use events for disc and destroy
+ common: reduce default max inline data size because of performance anomaly
+ common: dapls_evd_dto_wait() dbg message should print status and not errno
+ ucm, scm: exchange max_qp_rd_atom and limit outstanding requests
+ scm: retry socket connect on ECONNREFUSED under heavy load
+ common: qp modify RTR using wrong ep attribute parameter for dest_rd_atomic
+
+ Release 2.0.31 fixes (OFED 1.5.3 RC1):
+
+ common: clean up build warning for unused variable event_ptr
+ scm, ucm: set RAI_NOROUTE flag with rdma_getaddrinfo() call to avoid blocking.
+ cma: definition for dapl_sp_remove_ep() is missing in cm.c
+ libdat: static provider entries created for local SR database not freed
+ libdat: memory leak in static registration during parsing
+ common: increase default IB inline send threshold to 400
+ common cq: a mixup of errno and the -1 return from poll in dapls_wait_comp_channel
+ ucm: release UD cm objects after AH is exchanged to avoid duplicate request drops
+ ucm: decrease timeout retry count for disconnect requests
+ ucm: hold lock when sending cm_msgs to sync timer start with packet send
+ ucm: add debugging to include process id for better scale up debug aids
+ cma: disconnect can block for excessive times waiting for rdma_cm DREP timeout
+ ucm: configure the recv channel FD to non-blocking
+ windows: Missing librdmacm include path for build
+ debug build: only timestamp if sending to stdout to avoid performance hit
+ common: print out errors on free build and not just debug builds
+ cma: fix debug build issue
+ scm, ucm: MPI spawn test on oversubcribed server taking excessive time to complete
+ common: add high resolution time stamps and thread id to sdtout debug logs
+ common: modify debug in dat_evd_dequeue to reduce noise, only output on non-empty
+ cma: rdma_destroy_id called twice during device open bind error
+ common: dat_evd_dequeue (poll_cq) fails with invalid parameter after EP (qp) free
+ ucm: allow configuration of CM burst (signal) threshold on posting
+ cma: fix debug build
+ windows: debug version of windows does not build.
+ Allow DAPL out of band connection models to use ibacm to obtain path record data.
+ ucm: add missing map file for UCM provider
+ ibal: delay QP transition during disconnect phase
+ Revert "ibal: delay QP transition during disconnect phase"
+ ibal: delay QP transition during disconnect phase
+ common: restructure EVD processing to handle EP destruction phase
+ ibal: sync QP destruction and device close
+ ucm: remove unnecessary debug warning in async callback
+
+ v1.2 Package:
+
+ Release 1.2.19 fixes (OFED 1.5.2 GA):
+
+ common, cma: disconnect and cleanup CR linkings after DTO error on EP
+ common: race conditions with DTO error, disconnect and dapl_reset_ep
+ common: add new dapl_os_sleep_usec() function
+ configure: need a false conditional for verbs attr.link_layer member check
+ config: add conditional check for new verbs port_attr.link_layer
+ cma, scm: new provider entries for Mellanox RDMA over Ethernet device for uDAPL v1.2
+ cma: memory leak of verbs CQ and completion channels created during dat_ia_open
+ cma: memory leak of FD's (pipe) created during dat_evd_create
+
+ ----------------
+
+ * BKM for running new DAPL library on your cluster without any impact on existing OFED installation:
+
+ Note: example for user /home/ardavis, (assumes /home/ardavis is exported) and MLX4 adapter, port 1
+
+ Download latest 2.x package: http://www.openfabrics.org/downloads/dapl/dapl-2.0.32.tar.gz
+
+ untar in /home/ardavis
+ cd /home/ardavis/dapl-2.0.25
+ ./configure && make (build on node with OFED 1.3 or higher installed, dependency on verb/rdma_cm libraries)
+
+ create /home/ardavis/dat.conf with following 3 lines. (entries with path to new libraries):
+
+ ofa-v2-ib0 u2.0 nonthreadsafe default /home/ardavis/dapl-2.0.32/dapl/udapl/.libs/libdaplcma.so.1 dapl.2.0 "ib0 0" ""
+ ofa-v2-mlx4_0-1 u2.0 nonthreadsafe default /home/ardavis/dapl-2.0.32/dapl/udapl/.libs/libdaploscm.so.2 dapl.2.0 "mlx4_0 1" ""
+ ofa-v2-mlx4_0-1u u2.0 nonthreadsafe default /home/ardavis/dapl-2.0.32/dapl/udapl/.libs/libdaploucm.so.2 dapl.2.0 "mlx4_0 1" ""
+
+ Run uDAPL application or an MPI that uses uDAPL, with (assuming MLX4 connectx adapters) following:
+
+ setenv DAT_OVERRIDE=/home/ardavis/dat.conf
+ setenv LD_LIBRARY_PATH=/home/ardavis/dapl-2.0.32/dapl/udapl/.libs:$LD_LIBRARY_PATH
+
+ If running Intel MPI and uDAPL socket cm, set the following:
+
+ setenv I_MPI_DEVICE=rdssm:ofa-v2-mlx4_0-1
+
+ or if running Intel MPI and uDAPL IB UD cm, set the following:
+
+ setenv I_MPI_DEVICE=rdssm:ofa-v2-mlx4_0-1u
+
+ or if running Intel MPI and uDAPL rdma_cm, set the following:
+
+ setenv I_MPI_DEVICE=rdssm:ofa-v2-ib0
+
+
+--- HISTORY -----------
+
+ OFED 1.5.1 RELEASE NOTES
+ uDAPL v1 (1.2.16-1) and v2 (2.0.27-1)
+
+ ----------------
+
+ * New Features (v2 only) - UCM provider with IB UD based CM per process.
+ More scalable then rdma_cm (cma) or socket cm (scm).
+ ----------------
+
+ * Bug Fixes
+
+ V2.0 Package
+
Release 2.0.27
windows: add scm makefile
windows does not require rdma_cma_abi.h, move the include from common code
--
1.7.3
More information about the ewg
mailing list