[ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network
Yicheng Jia
YJia at tmriusa.com
Thu Aug 21 13:02:43 PDT 2008
> In the past i wrote even more complicated flows than this when the SM
> was down...
Can you point to me where it is?
Thanks!
Yicheng
Dotan Barak <dotanba at gmail.com>
08/21/2008 02:52 PM
To
Yicheng Jia <YJia at tmriusa.com>
cc
general at lists.openfabrics.org, Hal Rosenstock <hal.rosenstock at gmail.com>
Subject
Re: [ofa-general] minimum sw components requirement for driver/opensm in a
single unmanaged switch network
Yicheng Jia wrote:
>
> My operation is quite simple: connect QPs and do RDMA read/write. In
> this case, the opensm is not in need when the subnet is up, correct?
Basically yes, but it depends on how you connect the QPs ...
In the past i wrote even more complicated flows than this when the SM
was down...
If you'll connect the QPs using the sockets and you won't depend on
other ULP (such as IPoIB, SDP or any other) you will be fine ..
Dotan
>
> Thanks!
> Yicheng
>
>
>
> *"Dotan Barak" <dotanba at gmail.com>*
>
> 08/21/2008 02:33 PM
>
>
> To
> "Yicheng Jia" <YJia at tmriusa.com>
> cc
> "Hal Rosenstock" <hal.rosenstock at gmail.com>,
> general at lists.openfabrics.org
> Subject
> Re: [ofa-general] minimum sw components requirement for
driver/opensm
> in a single unmanaged switch network
>
>
>
>
>
>
>
>
>
> On Thu, Aug 21, 2008 at 10:16 PM, Yicheng Jia <YJia at tmriusa.com> wrote:
> >
> > Hi Hal,
> >
> > Can opensm just run once? When the subnet is up, it can exit assume
> that no
> > change will be made in the subnet.
> >
> Yes, depend on the serives that you will need/use.
>
> For example: if you use operations that requires SA query, you must
> have a live SM.
>
> If you will connect the QPs in the subnet by yourself (for example,
> using socket) you can manage without a live SM in the subnet ...
>
> Dotan
> > Thanks!
> > Yicheng
> >
> >
> >
> > "Hal Rosenstock" <hal.rosenstock at gmail.com>
> >
> > 07/10/2008 09:15 PM
> >
> > To
> > "Yicheng Jia" <YJia at tmriusa.com>
> > cc
> > "Jim Mott" <jim at mellanox.com>, general at lists.openfabrics.org
> > Subject
> > Re: [ofa-general] minimum sw components requirement for
> driver/opensm in a
> > single unmanaged switch network
> >
> >
> >
> >
> > On Thu, Jul 10, 2008 at 7:39 PM, Yicheng Jia <YJia at tmriusa.com> wrote:
> >>
> >>> If you want to avoid all the SM stuff, and are willing to program
the
> >>> switches directly (a few mads)
> >>
> >> Is it done by opensm?
> >
> > Yes.
> >
> >> What information should be set up in the switch by
> >> opensm?
> >
> > Things like the PortInfos and LFT. See IBA spec vol 1 14.2.5
> >
> >>> Then to figure out QP connections, you just use a function of 3
> >>> parameters:
> >>> my_qp_num = fn_sqp(my_node, target_node, qp_num)
> >>> target_qp_num = fn_tqp(my_node, target_node, qp_num)
> >>> Where qp_num is a small number between 0 and the maximum number of
QPs
> >>> you
> >>> need active between any 2 endpoints.
> >>
> >> Can the qp_num be manually assigned?
> >> Does it need opensm be involved?
> >
> > SM has nothing to do with QP numbers.
> >
> >>> If it works, you are done. If not, reset, up, wait for him to
connect
> >>> and
> >>> send something to you.
> >>
> >> Is it reliable? I mean the QPs connection will keep alive during
> the QPs
> >> lifecycle?
> >
> > For one thing, SM needs to try to keep ports at active.
> >
> > -- Hal
> >
> >> Best,
> >> Yicheng
> >>
> >>
> >>
> >> "Jim Mott" <jim at mellanox.com>
> >>
> >> 07/10/2008 04:17 PM
> >>
> >> To
> >> "Yicheng Jia" <YJia at tmriusa.com>, <general at lists.openfabrics.org>
> >> cc
> >> Subject
> >> RE: [ofa-general] minimum sw components requirement for
> driver/opensm in a
> >> single unmanaged switch network
> >>
> >>
> >>
> >>
> >> If you want to avoid all the SM stuff, and are willing to program the
> >> switches directly (a few mads), then I've used schemes like:
> >>
> >> Node LID=base + (switch port * constant) (base=0, constant = 1 works)
> >>
> >> Then to figure out QP connections, you just use a function of 3
> >> parameters:
> >> my_qp_num = fn_sqp(my_node, target_node, qp_num)
> >> target_qp_num = fn_tqp(my_node, target_node, qp_num)
> >> Where qp_num is a small number between 0 and the maximum number of
> QPs you
> >> need active between any 2 endpoints.
> >>
> >> With the above scheme, you know your node_id (switch port number),
your
> >> lid,
> >> the lid of the target node, and the QPs on both sides. From there
> on, it
> >> is clear sailing. You don't even need to send MADs; just
> transition the
> >> QP
> >> up and try and use it. If it works, you are done. If not, reset,
up,
> >> wait
> >> for him to connect and send something to you. A little timer to
> make sure
> >> everybody retries once in awhile and what can go wrong?
> >>
> >> Jim
> >> From: general-bounces at lists.openfabrics.org
> >> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Yicheng
Jia
> >> Sent: Thursday, July 10, 2008 2:59 PM
> >> To: general at lists.openfabrics.org
> >> Subject: [ofa-general] minimum sw components requirement for
> driver/opensm
> >> in a single unmanaged switch network
> >>
> >>
> >> Hi Folks,
> >>
> >> I have a IB network which consists of only a single unmanaged
> switch, all
> >> end nodes connecting with the switch only need to do RDMA read/write
> >> operation with each other. My question is, what are the indispensable
> >> modules in driver's core and opensm that make the network up and run?
> >>
> >> I've been using only ib_mad module in driver's core with a managed
> switch
> >> before, and the network works fine. So I assume that only the ib_mad
> >> module
> >> in driver's core and SM in opensm are mandatory in my network. The
LIDs
> >> are
> >> assigned by them. The SA and CM modules are not useful in my case. Am
I
> >> right?
> >>
> >> I need to minimize driver and opensm to fit them in my network, the
HCA
> >> driver is mthca.
> >>
> >> Best,
> >> Yicheng
> >>
> >>
>
_____________________________________________________________________________
> >> Scanned by IBM Email Security Management Services powered by
> MessageLabs.
> >> For more information please visit http://www.ers.ibm.com
> >>
> >>
>
_____________________________________________________________________________
> >>
> >>
> >>
>
_____________________________________________________________________________
> >> Scanned by IBM Email Security Management Services powered by
> MessageLabs.
> >> For more information please visit
> <http://www.ers.ibm.com/>http://www.ers.ibm.com
> >>
> >>
>
_____________________________________________________________________________
> >>
> >>
> >>
>
_____________________________________________________________________________
> >> Scanned by IBM Email Security Management Services powered by
> MessageLabs.
> >> For more information please visit
> <http://www.ers.ibm.com/>http://www.ers.ibm.com
> >>
> >>
>
_____________________________________________________________________________
> >>
> >> _______________________________________________
> >> general mailing list
> >> general at lists.openfabrics.org
> >>
> <http://www.ers.ibm.com/
>http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>
> >> To unsubscribe, please visit
> >> http://openib.org/mailman/listinfo/openib-general
> >>
> >
> >
>
_____________________________________________________________________________
> > Scanned by IBM Email Security Management Services powered by
> MessageLabs.
> > For more information please visit http://www.ers.ibm.com
> >
>
_____________________________________________________________________________
> >
> >
> >
>
_____________________________________________________________________________
> > Scanned by IBM Email Security Management Services powered by
> MessageLabs.
> > For more information please visit
> <http://www.ers.ibm.com/>http://www.ers.ibm.com
> >
>
_____________________________________________________________________________
> >
> > _______________________________________________
> > general mailing list
> > general at lists.openfabrics.org
> >
> <http://www.ers.ibm.com/
>http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >
> > To unsubscribe, please visit
> > http://openib.org/mailman/listinfo/openib-general
> >
>
>
_____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by
> MessageLabs. For more information please visit http://www.ers.ibm.com
>
_____________________________________________________________________________
> <http://www.ers.ibm.com/>
>
>
_____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by
> MessageLabs. For more information please visit http://www.ers.ibm.com
>
_____________________________________________________________________________
_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs.
For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________
_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080821/5ddc2074/attachment.html>
More information about the general
mailing list