[ofa-general] Re: [PATCHv2] opensm/lash: Set minimum VL for LASH to use

[Hal Rosenstock]

Hi Sasha,

On Mon, Jul 20, 2009 at 7:57 AM, Sasha Khapyorsky<sashak at voltaire.com> wrote:
> Hi Hal,
>
> On 11:46 Mon 06 Jul     , Hal Rosenstock wrote:
>>
>> diff --git a/opensm/opensm/osm_ucast_lash.c b/opensm/opensm/osm_ucast_lash.c
>> index 12b5e34..3db75a0 100644
>> --- a/opensm/opensm/osm_ucast_lash.c
>> +++ b/opensm/opensm/osm_ucast_lash.c
>> @@ -478,7 +478,7 @@ static void balance_virtual_lanes(lash_t * p_lash, unsigned lanes_needed)
>>       cdg_vertex_t ****cdg_vertex_matrix = p_lash->cdg_vertex_matrix;
>>       int *num_mst_in_lane = p_lash->num_mst_in_lane;
>>       int ***virtual_location = p_lash->virtual_location;
>> -     int min_filled_lane, max_filled_lane, trials;
>> +     int min_filled_lane, max_filled_lane, trials, max_vl;
>>       int old_min_filled_lane, old_max_filled_lane, new_num_min_lane,
>>           new_num_max_lane;
>>       unsigned int i, j;
>> @@ -486,9 +486,11 @@ static void balance_virtual_lanes(lash_t * p_lash, unsigned lanes_needed)
>>       int next_switch2, output_link2;
>>       int stop = 0, cycle_found;
>>       int cycle_found2;
>> +     unsigned start_vl = p_lash->p_osm->subn.opt.lash_start_vl;
>>
>> -     max_filled_lane = 0;
>> -     min_filled_lane = lanes_needed - 1;
>> +     max_filled_lane = start_vl;
>> +     max_vl = lanes_needed + start_vl;
>> +     min_filled_lane = max_vl - 1;
>>
>>       trials = num_mst_in_lane[max_filled_lane];
>>       if (lanes_needed == 1)
>> @@ -590,7 +592,7 @@ static void balance_virtual_lanes(lash_t * p_lash, unsigned lanes_needed)
>>               new_num_min_lane = MAX_INT;
>>               new_num_max_lane = 0;
>>
>> -             for (i = 0; i < lanes_needed; i++) {
>> +             for (i = start_vl; i < max_vl; i++) {
>>
>>                       if (num_mst_in_lane[i] < new_num_min_lane) {
>>                               new_num_min_lane = num_mst_in_lane[i];
>> @@ -674,11 +676,12 @@ static void free_lash_structures(lash_t * p_lash)
>>       unsigned int i, j, k;
>>       unsigned num_switches = p_lash->num_switches;
>>       osm_log_t *p_log = &p_lash->p_osm->log;
>> +     unsigned start_vl = p_lash->p_osm->subn.opt.lash_start_vl;
>>
>>       OSM_LOG_ENTER(p_log);
>>
>>       /* free cdg_vertex_matrix */
>> -     for (i = 0; i < p_lash->vl_min; i++) {
>> +     for (i = start_vl; i < p_lash->vl_min; i++) {
>>               for (j = 0; j < num_switches; j++) {
>>                       for (k = 0; k < num_switches; k++)
>>                               if (p_lash->cdg_vertex_matrix[i][j][k])
>> @@ -715,13 +718,14 @@ static int init_lash_structures(lash_t * p_lash)
>>       osm_log_t *p_log = &p_lash->p_osm->log;
>>       int status = 0;
>>       unsigned int i, j, k;
>> +     unsigned start_vl = p_lash->p_osm->subn.opt.lash_start_vl;
>>
>>       OSM_LOG_ENTER(p_log);
>>
>>       /* initialise cdg_vertex_matrix[num_switches][num_switches][num_switches] */
>>       p_lash->cdg_vertex_matrix =
>> -         (cdg_vertex_t ****) malloc(vl_min * sizeof(cdg_vertex_t ****));
>> -     for (i = 0; i < vl_min; i++) {
>> +         (cdg_vertex_t ****) calloc(vl_min, sizeof(cdg_vertex_t ****));
>
> You don't need to use calloc() when memory initialization is not
> required - malloc() is always faster.

I restored the mallocs in the next version of this patch.

> Wouldn't it be better instead of allocating extra memory for "disabled"
> (0 - lash_vl_start) VLs range and tracking 'max_vl' in many places just
> to setup lash->vl_min at end of discover_network_properties() as vl_min -
> lash_vl_start and to increase routing_table[i].lane value by
> lash_vl_start at end of lash cycle?

Something like that could be done. Does it save significant memory
(haven't looked yet to see exactly) ? OK as a subsequent patch to this
change ?

-- Hal

> Sasha
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>