[ofa-general] Performance of UDAPL RDMA vs IB verbs
Chuck Hartley
hartlch14 at gmail.com
Fri Feb 15 13:09:33 PST 2008
Well I expect that is the problem then. I see the same problem when I don't
run the OpenSM at all and just use the one running on the blade server. All
those are SDR HCAs and that must be all that it will negotiate for. I'll
have to figure out how to turn it off and just run the OpenSM one.
On Fri, Feb 15, 2008 at 4:04 PM, Hal Rosenstock <hrosenstock at xsigo.com>
wrote:
> On Fri, 2008-02-15 at 15:51 -0500, Chuck Hartley wrote:
> > Would another (non-OpenSM) SM running on another switch mess things
> > up? We have an IBM blade server with its own IB switch attached to
> > our main switch. Detaching that connection made the problem go away.
> > Reattaching it makes the problem come back. Do I have to configure
> > something differently in opensmd.conf?
>
> You're not supposed to run different SM "flavors" in the same subnet so
> if there is some other SM running one of them needs to be disabled. You
> can run multiple instances of the same SM flavor though.
>
> -- Hal
>
> > On Fri, Feb 15, 2008 at 3:33 PM, Hal Rosenstock
> > <hrosenstock at xsigo.com> wrote:
> > On Fri, 2008-02-15 at 15:26 -0500, Chuck Hartley wrote:
> > > I direct connected the two machines instead of running them
> > to the
> > > switch and I get good BW numbers. What does that indicate?
> >
> >
> > Maybe you have a degraded link in your subnet.
> >
> >
> > >
> > >
> > > On Fri, Feb 15, 2008 at 3:05 PM, Chuck Hartley
> > <hartlch14 at gmail.com>
> > > wrote:
> > > Nope - none of those. The closest thing is
> > osm_pr_rcv_init
> > > right when the SM starts up.
> > >
> > > BTW, we have OpenSM Rev:openib-3.0.14
> > >
> > >
> > > On Fri, Feb 15, 2008 at 2:56 PM, Hal Rosenstock
> > > <hrosenstock at xsigo.com> wrote:
> > > On Fri, 2008-02-15 at 14:47 -0500, Chuck
> > Hartley
> > > wrote:
> > > > Yes, but I don't see anything in the log
> > file tagged
> > > with anything
> > > > like that. Is there some other string I
> > can search
> > > on to locate the
> > > > request/response messages?
> > >
> > >
> > > Maybe search for osm_pr_rcv_respond ?
> > >
> > >
> > > > What osm_xxx function would be sending the
> > > request/responses? Or what
> > > > is the message or signal called - I see
> > various ones
> > > starting with
> > > > OSM_
> > > >
> > > >
> > > >
> > > > On Fri, Feb 15, 2008 at 1:52 PM, Hal
> > Rosenstock
> > > > <hrosenstock at xsigo.com> wrote:
> > > > On Fri, 2008-02-15 at 13:46 -0500,
> > Chuck
> > > Hartley wrote:
> > > > > Yes, we are using OpenSM and I
> > found where
> > > to set the -V
> > > > switch
> > > > > in /etc/opensm.conf. I started
> > opensmd
> > > service on the
> > > > current node
> > > > > and looked at the log file it
> > created. I
> > > have no idea what
> > > > I am
> > > > > looking at / for... there are
> > two items
> > > that stand out -
> > > > PortInfo
> > > > > dumps and SMP dumps. What is it
> > I am
> > > looking for or what
> > > > can I post
> > > > > here?
> > > >
> > > >
> > > > SA PathRecord requests/responses
> > to your end
> > > node.
> > > >
> > > >
> > > > > On Fri, Feb 15, 2008 at 12:47
> > PM, Hal
> > > Rosenstock
> > > > > <hrosenstock at xsigo.com> wrote:
> > > > > On Fri, 2008-02-15 at
> > 09:30 -0800,
> > > Hal Rosenstock
> > > > wrote:
> > > > > > On Fri, 2008-02-15 at
> > 09:23
> > > -0800, Arlin Davis
> > > > wrote:
> > > > > > > I would concentrate
> > on the
> > > path record
> > > > information
> > > > > returned
> > > > > > > from the SA and
> > compare
> > > against the straight
> > > > verbs test
> > > > > > > configuration.
> > > > > > >
> > > > > > > Hal/Sean, is there
> > an easy way
> > > to see path
> > > > record
> > > > > information
> > > > > > > from the query?
> > > > > >
> > > > > > Most SMs have a way to
> > display
> > > the PathRecord
> > > > responses
> > > > > being returned
> > > > > > by the SA. Would that
> > work ? If
> > > OpenSM is being
> > > > used, use -V
> > > > > on the
> > > > > > command line for this.
> > Contact
> > > your vendor if a
> > > > vendor
> > > > > specific SM is
> > > > > > being used and this
> > technique
> > > would work for your
> > > > purposes.
> > > > >
> > > > >
> > > > > Another alternative
> > would be to
> > > use madeye on the
> > > > end node but
> > > > > I don't
> > > > > think there's much
> > decode there so
> > > that would need
> > > > to be done
> > > > > by "hand".
> > > > >
> > > > > -- Hal
> > > > >
> > > > > >
> > > > > > -- Hal
> > > > > >
> > > > > > > -arlin
> > > > > > >
> > > > > > >
> > > > > >
> > >
> > _______________________________________________
> > > > > > general mailing list
> > > > > >
> > general at lists.openfabrics.org
> > > > > >
> > > http://lists.openfabrics.org/cgi-
> > > > >
> > bin/mailman/listinfo/general
> > > > > >
> > > > > > To unsubscribe, please
> > visit
> > > > >
> > > http://openib.org/mailman/listinfo/openib-
> > general
> > > > >
> > >
> > _______________________________________________
> > > > > general mailing list
> > > > >
> > general at lists.openfabrics.org
> > > > >
> > http://lists.openfabrics.org/cgi-
> > > > bin/mailman/listinfo/general
> > > > >
> > > > > To unsubscribe, please
> > visit
> > > > >
> > > http://openib.org/mailman/listinfo/openib-
> > general
> > > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080215/0afe25b5/attachment.html>
More information about the general
mailing list