***SPAM*** Re: [ofa-general] ***SPAM*** ibdm network topology format

[Hal Rosenstock]

Sasha,

On Wed, Oct 1, 2008 at 10:24 PM, Sasha Khapyorsky <sashak at voltaire.com> wrote:
> Hi Manesh,
>
> On 23:38 Wed 01 Oct     , Sasha Khapyorsky wrote:
>>
>> On 14:37 Wed 01 Oct     , Keshetti Mahesh wrote:
>> >
>> > I have tried running 'ibdiagnet' on ibsim after exporting SIM_HOST
>> > environment variable to some host name. But still 'ibdiagnet' failed to
>> > discover the topology. See the below output.
>>
>> Hmm, I have similar results now. Will try to investigate deeper...
>
> Right now I found two issues:
>
> 1. ibis is linked statically with libosmvendor libibumad. libumad2sim.so
> is loaded (preloaded) dynamically and requires libibumad.so as well. As
> result we have two instances of libibumad - only one is initialized
> properly so only one instance of libibumad has flag new_user_mad_api set.
> As result libibumad(s) used by libosmvendor and by libumad2sim use
> different user_mad header formats.
>
> It may be solved if we will not use static linkage with ibis. Patch like
> this seems work for me:
>
> diff --git a/ibis/src/Makefile.am b/ibis/src/Makefile.am
> index e0b512f..a52e67f 100644
> --- a/ibis/src/Makefile.am
> +++ b/ibis/src/Makefile.am
> @@ -74,8 +74,6 @@ LDADD = $(OSM_LDFLAGS)
>
>  ibis_SOURCES = ibissh_wrap.cpp
>
> -ibis_LDFLAGS = -static
> -# note the order of the libraries does matter as we static link
>  ibis_LDADD = -libiscom $(OSM_LDFLAGS) $(TCL_LIBS)
>
>
> @@ -153,7 +151,8 @@ EXTRA_DIST = swig_extended_obj.c fixSwigWrapper pkgIndex.tcl \
>         git_version.h
>
>  # we do not want the temporary and the archive libs installed:
> -install-libLTLIBRARIES:
> +# then objects should be linked into program
> +#install-libLTLIBRARIES:
>
>  # this actually over write the lib install
>  install-exec-am: install-binPROGRAMS
>
>
> 2. ibis doesn't register class 0x81 - SM direct routed, only SM lid
> routed (0x1). In comment in ibutils/ibis/src/ibsm.c line 118 is stated:
>
>  /* no need to bind the Directed Route class as it will automatically
>     be handled by the osm_vendor_bind if asked for LID route */
>
> As far as I can see in osm_vendor_bind() it is not (but it is in
> opposite order - when class 0x81 is registered class 0x1 will be
> registered too).

Yes that is what osm_vendor_ibumad.c:osm_vendor_bind does.

So either ibdiagnet needs to register 0x81 r.t.1 or
osm_vendor_ibumad.c:osm_vendor_bind needs to be "symmetric" in terms
of registering the other SM class when only one is requested. This is
a minor change in the underlying semantics. [Popping up a level in
terms of this, (other than applications taking advantage of this
"feature",) I'm not sure why the vendor layer should assume that just
because one SM class is requested, the other should be too].  I just
looked and the latter appears to be consistent with the other vendor
layers. I think either solution will work. Your solution below also
looks like it would work but don't that should be done in a sim layer.

> Somehow it works without ibsim - so I suspect user_mad handles it.
>
> (Hal, could you clarify?)

The kernel (user_mad/mad) does not change the requested registrations
but I'm not sure I understand the question you are asking to be
clarified. Is that what you're asking ?

-- Hal

> Assuming this is the case, then patch to umad2sim (ibsim) like this can
> help (helps for me):
>
> diff --git a/umad2sim/umad2sim.c b/umad2sim/umad2sim.c
> index 646cde2..a6196f1 100644
> --- a/umad2sim/umad2sim.c
> +++ b/umad2sim/umad2sim.c
> @@ -407,7 +408,7 @@ static ssize_t umad2sim_read(struct umad2sim_dev *dev, void *buf, size_t count)
>                mgmt_class = 0;
>        }
>
> -       umad->agent_id = dev->agent_idx[mgmt_class];
> +       umad->agent_id = dev->agent_idx[mgmt_class&(~0x80)];
>        umad->status = ntohl(req.status);
>        umad->timeout_ms = 0;
>        umad->retries = 0;
>
>
> But I'm not sure yet that it is proper solution (up to class registration
> issue clarification).
>
> Sasha
>