[ofa-general] [RFC] Fat-Tree upgrades

[Nicolas Morey Chaisemartin]

Yevgeny Kliteynik wrote:
> The speed penalty was my other concern.
> Will be happy to see the comparison of the calculation time with and w/o
> the sorting once its ready.
> 

I finally managed to achieve "good" performances.
I mesured only the routing time  using clock() (see patch at the end of the mail) and this command:
 for i in `seq 1 5`; do  opensm -o -a /home_nfs/moreyn/workspace/git/ibsim-t100/config/ftree_root_guids-r8.txt   --console local -R ftree; done   | grep "Routing time"
Network a bit under 4k nodes in ibsim. Topology is a well balanced fat tree, though not completely regular (hence the root guid file).

On master (1f5fdf3dfc69733a427520198bcbdd03645bb326):
Routing time: 1730000
Routing time: 1710000
Routing time: 1700000
Routing time: 1730000
Routing time: 1720000

On my branch with enhanced fat-tree:
Routing time: 1810000
Routing time: 1800000
Routing time: 1790000
Routing time: 1800000
Routing time: 1810000

So we have about 5% loss in routing time which is not so bad.
However, as I had to modify my sorting algorithms and tweak a lots of things, I'm still running some non regression tests.
Patches shall be posted later in the afternoon if they still work fine.

Nicolas




______________________
commit 7db9f3a218d2460fc9826335efdab16cbf73bfb1
Author: Nicolas Morey-Chaisemartin <nicolas.morey-chaisemartin at ext.bull.net>
Date:   Thu Mar 5 10:00:13 2009 +0100

    Added clock tools for timing

diff --git a/opensm/opensm/osm_ucast_ftree.c b/opensm/opensm/osm_ucast_ftree.c
index fffd519..489b788 100644
--- a/opensm/opensm/osm_ucast_ftree.c
+++ b/opensm/opensm/osm_ucast_ftree.c
@@ -51,6 +51,7 @@
 #include <complib/cl_debug.h>
 #include <opensm/osm_opensm.h>
 #include <opensm/osm_switch.h>
+#include <time.h>
 
 /*
  * FatTree rank is bounded between 2 and 8:
@@ -76,6 +77,9 @@ typedef enum {
        FTREE_DIRECTION_UP
 } ftree_direction_t;
 
+clock_t routing_time=0;
 /***************************************************
  **
  **  Forward references
@@ -3861,7 +3865,7 @@ static int __osm_ftree_do_routing(IN void *context)
                status = -1;
                goto Exit;
        }
-
+       routing_time=clock();
        OSM_LOG(&p_ftree->p_osm->log, OSM_LOG_VERBOSE,
                "Starting FatTree routing\n");
 
@@ -3876,11 +3880,13 @@ static int __osm_ftree_do_routing(IN void *context)
        OSM_LOG(&p_ftree->p_osm->log, OSM_LOG_VERBOSE,
                "Filling switch forwarding tables for switch-to-switch paths\n");
        __osm_ftree_fabric_route_to_switches(p_ftree);
-
+       routing_time=clock()-routing_time;
        /* for each switch, set its fwd table */
        cl_qmap_apply_func(&p_ftree->sw_tbl, __osm_ftree_set_sw_fwd_table,
                           (void *)p_ftree);
 
+       printf("Routing time: %lu\n",routing_time);
        /* write out hca ordering file */
        __osm_ftree_fabric_dump_hca_ordering(p_ftree);