[openib-general] Some opensm/osm_vl15intf.c questions

[Hal Rosenstock]

Hi Eitan,

In chasing an issue with a trap repress not being sent in a certain
scenario, I stumbled across the following questions about
opensm/osm_vl15intf.c.

1. osm_vl15_post increments qp0_mads_outstanding when a response is
expected (rfifo) and not when unsolicited (ufifo) (what appears to be
called unicasts):

osm_vl15_post:
  if( p_madw->resp_expected == TRUE )
  {
    cl_qlist_insert_tail( &p_vl->rfifo, (cl_list_item_t*)p_madw );
    cl_atomic_inc( &p_vl->p_stats->qp0_mads_outstanding );
  }
  else
  {
    cl_qlist_insert_tail( &p_vl->ufifo, (cl_list_item_t*)p_madw );
  }

osm_vl15_shutdown retires all outstanding MADs as follows:

osm_vl15_shutdown:
  while ( p_madw != (osm_madw_t*)cl_qlist_end( &p_vl->ufifo ) )
  {
    if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) )
    {
      osm_log( p_vl->p_log, OSM_LOG_DEBUG,
               "osm_vl15_shutdown: "
               "Releasing Response p_madw = %p\n", p_madw );
    }

    osm_mad_pool_put( p_mad_pool, p_madw );
    cl_atomic_dec( &p_vl->p_stats->qp0_mads_outstanding );

    p_madw = (osm_madw_t*)cl_qlist_remove_head( &p_vl->ufifo );
  }

Either post should increment qp0_mads_outstanding for unsolicited or
shutdown shouldn't decrement it when removing from ufifo. If you agree,
which should it be ? 

2. In the case of a failure from osm_vendor_send, __osm_vl15_poller
decrements qp0_mads_outstanding regardless of whether a response is
expected. This is inconsistent with the increment. This leads me to
believe that this should also be incremented for unsolicited (unicasts)
as well as those for which responses are expected. Is this correct or am
I missing something ?

So my conclusion is that in osm_vl15_post, it should be:

  if( p_madw->resp_expected == TRUE )
  {
    cl_qlist_insert_tail( &p_vl->rfifo, (cl_list_item_t*)p_madw );
  }
  else
  {
    cl_qlist_insert_tail( &p_vl->ufifo, (cl_list_item_t*)p_madw );
  }
  cl_atomic_inc( &p_vl->p_stats->qp0_mads_outstanding );

If you agree, I will generate a patch for this. Thanks.

-- Hal