[ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network
Yicheng Jia
YJia at tmriusa.com
Wed Aug 27 22:22:00 PDT 2008
Yes. My basic idea is, the opensm set up the subnet during initialization,
it will report errors during this process. After the subnet is up, the
environment is fixed and stable. If some failure happens, opensm could be
used again to diagnose the failure. From my understanding, in this case,
the only work that opensm does after subnet is up is to log the status.
Thanks!
Yicheng
Hal Rosenstock <halr at obsidianresearch.com>
08/27/2008 05:55 PM
To
Yicheng Jia <YJia at tmriusa.com>
cc
Dotan Barak <dotanba at gmail.com>, general at lists.openfabrics.org
Subject
Re: [ofa-general] minimum sw components requirement for driver/opensm in a
single unmanaged switch network
Yicheng Jia wrote:
> My operation is quite simple: connect QPs and do RDMA read/write. In
this
> case, the opensm is not in need when the subnet is up, correct?
>
Is this a production subnet ? Do you need to deal with any failures ?
-- Hal
> Thanks!
> Yicheng
>
>
>
>
> "Dotan Barak" <dotanba at gmail.com>
> 08/21/2008 02:33 PM
>
> To
> "Yicheng Jia" <YJia at tmriusa.com>
> cc
> "Hal Rosenstock" <hal.rosenstock at gmail.com>,
general at lists.openfabrics.org
> Subject
> Re: [ofa-general] minimum sw components requirement for driver/opensm in
a
> single unmanaged switch network
>
>
>
>
>
>
> On Thu, Aug 21, 2008 at 10:16 PM, Yicheng Jia <YJia at tmriusa.com> wrote:
>
>> Hi Hal,
>>
>> Can opensm just run once? When the subnet is up, it can exit assume
that
>>
> no
>
>> change will be made in the subnet.
>>
>>
> Yes, depend on the serives that you will need/use.
>
> For example: if you use operations that requires SA query, you must
> have a live SM.
>
> If you will connect the QPs in the subnet by yourself (for example,
> using socket) you can manage without a live SM in the subnet ...
>
> Dotan
>
>> Thanks!
>> Yicheng
>>
>>
>>
>> "Hal Rosenstock" <hal.rosenstock at gmail.com>
>>
>> 07/10/2008 09:15 PM
>>
>> To
>> "Yicheng Jia" <YJia at tmriusa.com>
>> cc
>> "Jim Mott" <jim at mellanox.com>, general at lists.openfabrics.org
>> Subject
>> Re: [ofa-general] minimum sw components requirement for driver/opensm
in
>>
> a
>
>> single unmanaged switch network
>>
>>
>>
>>
>> On Thu, Jul 10, 2008 at 7:39 PM, Yicheng Jia <YJia at tmriusa.com> wrote:
>>
>>>> If you want to avoid all the SM stuff, and are willing to program the
>>>> switches directly (a few mads)
>>>>
>>> Is it done by opensm?
>>>
>> Yes.
>>
>>
>>> What information should be set up in the switch by
>>> opensm?
>>>
>> Things like the PortInfos and LFT. See IBA spec vol 1 14.2.5
>>
>>
>>>> Then to figure out QP connections, you just use a function of 3
>>>> parameters:
>>>> my_qp_num = fn_sqp(my_node, target_node, qp_num)
>>>> target_qp_num = fn_tqp(my_node, target_node, qp_num)
>>>> Where qp_num is a small number between 0 and the maximum number of
QPs
>>>> you
>>>> need active between any 2 endpoints.
>>>>
>>> Can the qp_num be manually assigned?
>>> Does it need opensm be involved?
>>>
>> SM has nothing to do with QP numbers.
>>
>>
>>>> If it works, you are done. If not, reset, up, wait for him to
connect
>>>> and
>>>> send something to you.
>>>>
>>> Is it reliable? I mean the QPs connection will keep alive during the
>>>
> QPs
>
>>> lifecycle?
>>>
>> For one thing, SM needs to try to keep ports at active.
>>
>> -- Hal
>>
>>
>>> Best,
>>> Yicheng
>>>
>>>
>>>
>>> "Jim Mott" <jim at mellanox.com>
>>>
>>> 07/10/2008 04:17 PM
>>>
>>> To
>>> "Yicheng Jia" <YJia at tmriusa.com>, <general at lists.openfabrics.org>
>>> cc
>>> Subject
>>> RE: [ofa-general] minimum sw components requirement for driver/opensm
>>>
> in a
>
>>> single unmanaged switch network
>>>
>>>
>>>
>>>
>>> If you want to avoid all the SM stuff, and are willing to program the
>>> switches directly (a few mads), then I've used schemes like:
>>>
>>> Node LID=base + (switch port * constant) (base=0, constant = 1 works)
>>>
>>> Then to figure out QP connections, you just use a function of 3
>>> parameters:
>>> my_qp_num = fn_sqp(my_node, target_node, qp_num)
>>> target_qp_num = fn_tqp(my_node, target_node, qp_num)
>>> Where qp_num is a small number between 0 and the maximum number of QPs
>>>
> you
>
>>> need active between any 2 endpoints.
>>>
>>> With the above scheme, you know your node_id (switch port number),
your
>>> lid,
>>> the lid of the target node, and the QPs on both sides. From there
on,
>>>
> it
>
>>> is clear sailing. You don't even need to send MADs; just transition
>>>
> the
>
>>> QP
>>> up and try and use it. If it works, you are done. If not, reset, up,
>>> wait
>>> for him to connect and send something to you. A little timer to make
>>>
> sure
>
>>> everybody retries once in awhile and what can go wrong?
>>>
>>> Jim
>>> From: general-bounces at lists.openfabrics.org
>>> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Yicheng
Jia
>>> Sent: Thursday, July 10, 2008 2:59 PM
>>> To: general at lists.openfabrics.org
>>> Subject: [ofa-general] minimum sw components requirement for
>>>
> driver/opensm
>
>>> in a single unmanaged switch network
>>>
>>>
>>> Hi Folks,
>>>
>>> I have a IB network which consists of only a single unmanaged switch,
>>>
> all
>
>>> end nodes connecting with the switch only need to do RDMA read/write
>>> operation with each other. My question is, what are the indispensable
>>> modules in driver's core and opensm that make the network up and run?
>>>
>>> I've been using only ib_mad module in driver's core with a managed
>>>
> switch
>
>>> before, and the network works fine. So I assume that only the ib_mad
>>> module
>>> in driver's core and SM in opensm are mandatory in my network. The
LIDs
>>> are
>>> assigned by them. The SA and CM modules are not useful in my case. Am
I
>>> right?
>>>
>>> I need to minimize driver and opensm to fit them in my network, the
HCA
>>> driver is mthca.
>>>
>>> Best,
>>> Yicheng
>>>
>>>
>>>
>
_____________________________________________________________________________
>
>>> Scanned by IBM Email Security Management Services powered by
>>>
> MessageLabs.
>
>>> For more information please visit http://www.ers.ibm.com
>>>
>>>
>>>
>
_____________________________________________________________________________
>
>>>
>>>
>
_____________________________________________________________________________
>
>>> Scanned by IBM Email Security Management Services powered by
>>>
> MessageLabs.
>
>>> For more information please visit http://www.ers.ibm.com
>>>
>>>
>>>
>
_____________________________________________________________________________
>
>>>
>>>
>
_____________________________________________________________________________
>
>>> Scanned by IBM Email Security Management Services powered by
>>>
> MessageLabs.
>
>>> For more information please visit http://www.ers.ibm.com
>>>
>>>
>>>
>
_____________________________________________________________________________
>
>>> _______________________________________________
>>> general mailing list
>>> general at lists.openfabrics.org
>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>>
>>> To unsubscribe, please visit
>>> http://openib.org/mailman/listinfo/openib-general
>>>
>>>
>>
>
_____________________________________________________________________________
>
>> Scanned by IBM Email Security Management Services powered by
>>
> MessageLabs.
>
>> For more information please visit http://www.ers.ibm.com
>>
>>
>
_____________________________________________________________________________
>
>>
>>
>
_____________________________________________________________________________
>
>> Scanned by IBM Email Security Management Services powered by
>>
> MessageLabs.
>
>> For more information please visit http://www.ers.ibm.com
>>
>>
>
_____________________________________________________________________________
>
>> _______________________________________________
>> general mailing list
>> general at lists.openfabrics.org
>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>
>> To unsubscribe, please visit
>> http://openib.org/mailman/listinfo/openib-general
>>
>>
>
>
_____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by
MessageLabs.
> For more information please visit http://www.ers.ibm.com
>
_____________________________________________________________________________
>
>
>
>
_____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by
MessageLabs. For more information please visit http://www.ers.ibm.com
>
_____________________________________________________________________________
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit
http://openib.org/mailman/listinfo/openib-general
_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs.
For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________
_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080828/9fb52bb8/attachment.html>
More information about the general
mailing list