<html>
<body>
<font size=3>At 03:53 PM 2/1/2008, Yicheng Jia wrote:<br><br>
</font><blockquote type=cite class=cite cite=""><tt>
<font face="Courier, Courier" size=2>> Not if the communication is
just within each subnet; only if it is<br>
> across subnets.</font></tt><font size=3> <br>
</font><tt><font face="Courier, Courier" size=2>It will cross the
subnets. For my case, each node has the need of sending message to other
nodes than its neighbor, e.g. A -> D or
E->B.</font></tt><font size=3> </font></blockquote><br>
IB is a point-to-point, switch technology. In a
point-to-point, two nodes can communicate via a pair of
ports. If you want to communicate across more than two nodes,
then you must use a switch. There is no support for a ring
topology ala FDDI / token ring. There is no work that will
help you implement a ring - a router topology to avoid a switch would
largely negate the performance benefits found in IB since now software
would be involved in every message. Further, you would need a
HCA port per communicating node which will cost quite a bit given the N-1
ports needed for point-to-point communication. The IB
architecture overview within the volume 1 specification provides all of
the details on what can be constructed if you want additional information
but as it stands, you'll need to purchase a switch.<br><br>
Mike<br><br>
<br>
<blockquote type=cite class=cite cite=""><tt>
<font face="Courier, Courier" size=2>Thanks!</font></tt><font size=3>
<br>
</font><tt><font face="Courier, Courier" size=2>Yicheng</font></tt>
<font size=3> <br><br>
<br><br>
</font><font size=1><b>Hal Rosenstock <hrosenstock@xsigo.com></b>
</font><font size=3><br><br>
</font><font size=1>02/01/2008 05:43 PM</font><font size=3> <br>
</font><div align="right"><font size=1>To<br>
</div>
Yicheng Jia <YJia@tmriusa.com></font><font size=3> <br>
</font><div align="right"><font size=1>cc<br>
</div>
general@lists.openfabrics.org, Roland Dreier
<rdreier@cisco.com></font><font size=3> <br>
</font><div align="right"><font size=1>Subject<br>
</div>
Re: [ofa-general] SA code?</font><font size=3> <br><br>
<br><br>
<br>
</font><tt><font face="Courier, Courier" size=2>On Fri, 2008-02-01 at
17:07 -0600, Yicheng Jia wrote:<br>
> <br>
> > Are you saying all messages are between adjacent neighbors on
the<br>
> ring ? <br>
> Yes. Say I have 6 nodes: A, B, C, D, E, F, then the ring topo will
be<br>
> A<->B, B<->C, C<->D, D<->E, E<->F,
F<->A, and there are totally 6<br>
> subnets. <br>
> <br>
> > If it is just A->B, B->C, ..., and never across the ring,
then this<br>
> can<br>
> > work but you need an SM instance per subnet.<br>
> Do I need a router per subnet as well? <br><br>
Not if the communication is just within each subnet; only if it is<br>
across subnets.<br><br>
> > If you need some router mode, then this is still a work in
progress<br>
> for<br>
> > InfiniBand although you might be able to get to a
proprietary<br>
> solution<br>
> > in the short term. <br>
> What version of OFED will have router mode available? <br><br>
There's already the beginnings of this but I don't know when a
complete<br>
solution would be available.<br><br>
-- Hal<br><br>
> Thanks! <br>
> Yicheng<br>
> <br>
> <br>
> <br>
> Hal Rosenstock<br>
> <hrosenstock@xsigo.com> <br>
> <br>
> 02/01/2008 04:45 PM <br>
> <br>
> <br>
>
To<br>
> Yicheng Jia<br>
> <YJia@tmriusa.com> <br>
>
cc<br>
> general@lists.openfabrics.org, Roland Dreier
<rdreier@cisco.com> <br>
>
Subject<br>
> Re: [ofa-general]<br>
> SA code?<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> Hi Yicheng,<br>
> <br>
> On Fri, 2008-02-01 at 16:38 -0600, Yicheng Jia wrote:<br>
> > <br>
> > Hi Hal, <br>
> > <br>
> > The only reason is cost down.<br>
> <br>
> Understood.<br>
> <br>
> > The cost of IB switch weighs too much in our product, it
seems the<br>
> > ring topology is the only way to go. There's no message across
the<br>
> > ring, all the communication occurs within the ring. <br>
> <br>
> Are you saying all messages are between adjacent neighbors on
the<br>
> ring ?<br>
> <br>
> > Would it cause the performance substantially go down? <br>
> <br>
> If it is just A->B, B->C, ..., and never across the ring, then
this<br>
> can<br>
> work but you need an SM instance per subnet.<br>
> <br>
> If you need some router mode, then this is still a work in
progress<br>
> for<br>
> InfiniBand although you might be able to get to a proprietary
solution<br>
> in the short term.<br>
> <br>
> -- Hal<br>
> <br>
> > Thanks!<br>
> > Yicheng <br>
> > <br>
> > <br>
> > <br>
> > Hal Rosenstock<br>
> > <hrosenstock@xsigo.com> <br>
> > <br>
> > 02/01/2008 04:25 PM <br>
> > <br>
> > <br>
>
>
To<br>
> > Yicheng Jia<br>
> > <YJia@tmriusa.com> <br>
>
>
cc<br>
> > Roland Dreier<br>
> > <rdreier@cisco.com>, general@lists.openfabrics.org <br>
> >
Subject<br>
> > Re: [ofa-general]<br>
> > SA code?<br>
> > <br>
> > <br>
> > <br>
> > <br>
> > <br>
> > <br>
> > <br>
> > <br>
> > On Fri, 2008-02-01 at 16:14 -0600, Yicheng Jia wrote:<br>
> > > <br>
> > > Hi Roland, <br>
> > > <br>
> > > Is there SA code available in OFED? I have a IB switch to
connect<br>
> > all<br>
> > > the mellanox HCAs, the SA is running on the switch's
firmware. My<br>
> > goal<br>
> > > is to skip the switch and use ring topology to connect the
HCA<br>
> with<br>
> > > each other. Is it doable in OFED? <br>
> > <br>
> > I don't understand what you're trying to accomplish with
this<br>
> > topology.<br>
> > In IB speak, each HCA <-> HCA link would be a separate
subnet. Do<br>
> you<br>
> > intend to forward across your ring ? If so, only some
"router" mode<br>
> > would allow that with the topology you propose.<br>
> > <br>
> > -- Hal<br>
> > <br>
> > > Thanks!<br>
> > > Yicheng<br>
> > > _______________________________________________<br>
> > > general mailing list<br>
> > > general@lists.openfabrics.org<br>
> > >
<a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general" eudora="autourl">
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general</a><br>
> > > <br>
> > > To unsubscribe, please visit<br>
> >
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a><br>
> > <br>
> > <br>
> ><br>
>
_____________________________________________________________________________<br>
> > Scanned by IBM Email Security Management Services powered
by<br>
> > MessageLabs. For more information please visit<br>
>
<a href="http://www.ers.ibm.com/" eudora="autourl">
http://www.ers.ibm.com</a><br>
> ><br>
>
_____________________________________________________________________________<br>
> > <br>
> <br>
> <br>
>
_____________________________________________________________________________<br>
> Scanned by IBM Email Security Management Services powered by<br>
> MessageLabs. For more information please visit
<a href="http://www.ers.ibm.com/" eudora="autourl">
http://www.ers.ibm.com</a><br>
>
_____________________________________________________________________________<br>
> <br><br>
<br>
_____________________________________________________________________________<br>
Scanned by IBM Email Security Management Services powered by MessageLabs.
For more information please visit
<a href="http://www.ers.ibm.com/" eudora="autourl">
http://www.ers.ibm.com</a><br>
_____________________________________________________________________________<br>
</font></tt><font size=3><br>
_______________________________________________<br>
general mailing list<br>
general@lists.openfabrics.org<br>
<a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general" eudora="autourl">
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general</a><br><br>
To unsubscribe, please visit
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a>
</font></blockquote></body>
</html>