<br><font size=2 face="sans-serif">I missed it. So it's better to keep
opensm running all the time?</font>
<br>
<br><font size=2 face="sans-serif">Thanks!</font>
<br><font size=2 face="sans-serif">Yicheng</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>Hal Rosenstock <halr@obsidianresearch.com></b>
</font>
<p><font size=1 face="sans-serif">08/28/2008 06:31 AM</font>
<td width=59%>
<table width=100%>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td><font size=1 face="sans-serif">Yicheng Jia <YJia@tmriusa.com></font>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td><font size=1 face="sans-serif">Dotan Barak <dotanba@gmail.com>,
general@lists.openfabrics.org</font>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td><font size=1 face="sans-serif">Re: [ofa-general] minimum sw components
requirement for driver/opensm in a single unmanaged switch network</font></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br>
<br><tt><font size=2>Yicheng Jia wrote:<br>
> Yes. My basic idea is, the opensm set up the subnet during initialization,
<br>
> it will report errors during this process. After the subnet is up,
the <br>
> environment is fixed and stable. If some failure happens, opensm could
be <br>
> used again to diagnose the failure. From my understanding, in this
case, <br>
> the only work that opensm does after subnet is up is to log the status.<br>
Wouldn't opensm also repair the failure if it could ?<br>
<br>
-- Hal<br>
> <br>
<br>
> Thanks!<br>
> Yicheng<br>
><br>
><br>
><br>
><br>
> Hal Rosenstock <halr@obsidianresearch.com> <br>
> 08/27/2008 05:55 PM<br>
><br>
> To<br>
> Yicheng Jia <YJia@tmriusa.com><br>
> cc<br>
> Dotan Barak <dotanba@gmail.com>, general@lists.openfabrics.org<br>
> Subject<br>
> Re: [ofa-general] minimum sw components requirement for driver/opensm
in a <br>
> single unmanaged switch network<br>
><br>
><br>
><br>
><br>
><br>
><br>
> Yicheng Jia wrote:<br>
> <br>
>> My operation is quite simple: connect QPs and do RDMA read/write.
In <br>
>> <br>
> this <br>
> <br>
>> case, the opensm is not in need when the subnet is up, correct?<br>
>><br>
>> <br>
> Is this a production subnet ? Do you need to deal with any failures
?<br>
><br>
> -- Hal<br>
><br>
> <br>
>> Thanks!<br>
>> Yicheng<br>
>><br>
>><br>
>><br>
>><br>
>> "Dotan Barak" <dotanba@gmail.com> <br>
>> 08/21/2008 02:33 PM<br>
>><br>
>> To<br>
>> "Yicheng Jia" <YJia@tmriusa.com><br>
>> cc<br>
>> "Hal Rosenstock" <hal.rosenstock@gmail.com>, <br>
>> <br>
> general@lists.openfabrics.org<br>
> <br>
>> Subject<br>
>> Re: [ofa-general] minimum sw components requirement for driver/opensm
in <br>
>> <br>
> a <br>
> <br>
>> single unmanaged switch network<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On Thu, Aug 21, 2008 at 10:16 PM, Yicheng Jia <YJia@tmriusa.com>
wrote:<br>
>><br>
>> <br>
>>> Hi Hal,<br>
>>><br>
>>> Can opensm just run once? When the subnet is up, it can exit
assume <br>
>>> <br>
> that <br>
> <br>
>> no<br>
>><br>
>> <br>
>>> change will be made in the subnet.<br>
>>><br>
>>><br>
>>> <br>
>> Yes, depend on the serives that you will need/use.<br>
>><br>
>> For example: if you use operations that requires SA query, you
must<br>
>> have a live SM.<br>
>><br>
>> If you will connect the QPs in the subnet by yourself (for example,<br>
>> using socket) you can manage without a live SM in the subnet ...<br>
>><br>
>> Dotan<br>
>><br>
>> <br>
>>> Thanks!<br>
>>> Yicheng<br>
>>><br>
>>><br>
>>><br>
>>> "Hal Rosenstock" <hal.rosenstock@gmail.com><br>
>>><br>
>>> 07/10/2008 09:15 PM<br>
>>><br>
>>> To<br>
>>> "Yicheng Jia" <YJia@tmriusa.com><br>
>>> cc<br>
>>> "Jim Mott" <jim@mellanox.com>, general@lists.openfabrics.org<br>
>>> Subject<br>
>>> Re: [ofa-general] minimum sw components requirement for driver/opensm
<br>
>>> <br>
> in <br>
> <br>
>> a<br>
>><br>
>> <br>
>>> single unmanaged switch network<br>
>>><br>
>>><br>
>>><br>
>>><br>
>>> On Thu, Jul 10, 2008 at 7:39 PM, Yicheng Jia <YJia@tmriusa.com>
wrote:<br>
>>><br>
>>> <br>
>>>>> If you want to avoid all the SM stuff, and are willing
to program the<br>
>>>>> switches directly (a few mads)<br>
>>>>><br>
>>>>> <br>
>>>> Is it done by opensm?<br>
>>>><br>
>>>> <br>
>>> Yes.<br>
>>><br>
>>><br>
>>> <br>
>>>> What information should be set up in the switch by<br>
>>>> opensm?<br>
>>>><br>
>>>> <br>
>>> Things like the PortInfos and LFT. See IBA spec vol 1 14.2.5<br>
>>><br>
>>><br>
>>> <br>
>>>>> Then to figure out QP connections, you just use a
function of 3<br>
>>>>> parameters:<br>
>>>>> my_qp_num = fn_sqp(my_node, target_node, qp_num)<br>
>>>>> target_qp_num = fn_tqp(my_node, target_node,
qp_num)<br>
>>>>> Where qp_num is a small number between 0 and the maximum
number of <br>
>>>>> <br>
> QPs<br>
> <br>
>>>>> you<br>
>>>>> need active between any 2 endpoints.<br>
>>>>><br>
>>>>> <br>
>>>> Can the qp_num be manually assigned?<br>
>>>> Does it need opensm be involved?<br>
>>>><br>
>>>> <br>
>>> SM has nothing to do with QP numbers.<br>
>>><br>
>>><br>
>>> <br>
>>>>> If it works, you are done. If not, reset, up,
wait for him to <br>
>>>>> <br>
> connect<br>
> <br>
>>>>> and<br>
>>>>> send something to you.<br>
>>>>><br>
>>>>> <br>
>>>> Is it reliable? I mean the QPs connection will keep alive
during the <br>
>>>><br>
>>>> <br>
>> QPs<br>
>><br>
>> <br>
>>>> lifecycle?<br>
>>>><br>
>>>> <br>
>>> For one thing, SM needs to try to keep ports at active.<br>
>>><br>
>>> -- Hal<br>
>>><br>
>>><br>
>>> <br>
>>>> Best,<br>
>>>> Yicheng<br>
>>>><br>
>>>><br>
>>>><br>
>>>> "Jim Mott" <jim@mellanox.com><br>
>>>><br>
>>>> 07/10/2008 04:17 PM<br>
>>>><br>
>>>> To<br>
>>>> "Yicheng Jia" <YJia@tmriusa.com>, <general@lists.openfabrics.org><br>
>>>> cc<br>
>>>> Subject<br>
>>>> RE: [ofa-general] minimum sw components requirement for
driver/opensm <br>
>>>><br>
>>>> <br>
>> in a<br>
>><br>
>> <br>
>>>> single unmanaged switch network<br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> If you want to avoid all the SM stuff, and are willing
to program the<br>
>>>> switches directly (a few mads), then I've used schemes
like:<br>
>>>><br>
>>>> Node LID=base + (switch port * constant) (base=0, constant
= 1 works)<br>
>>>><br>
>>>> Then to figure out QP connections, you just use a function
of 3<br>
>>>> parameters:<br>
>>>> my_qp_num = fn_sqp(my_node, target_node, qp_num)<br>
>>>> target_qp_num = fn_tqp(my_node, target_node, qp_num)<br>
>>>> Where qp_num is a small number between 0 and the maximum
number of QPs <br>
>>>> <br>
><br>
> <br>
>> you<br>
>><br>
>> <br>
>>>> need active between any 2 endpoints.<br>
>>>><br>
>>>> With the above scheme, you know your node_id (switch port
number), <br>
>>>> <br>
> your<br>
> <br>
>>>> lid,<br>
>>>> the lid of the target node, and the QPs on both
sides. From there <br>
>>>> <br>
> on, <br>
> <br>
>> it<br>
>><br>
>> <br>
>>>> is clear sailing. You don't even need to send MADs;
just transition <br>
>>>><br>
>>>> <br>
>> the<br>
>><br>
>> <br>
>>>> QP<br>
>>>> up and try and use it. If it works, you are done.
If not, reset, up,<br>
>>>> wait<br>
>>>> for him to connect and send something to you. A
little timer to make <br>
>>>><br>
>>>> <br>
>> sure<br>
>><br>
>> <br>
>>>> everybody retries once in awhile and what can go wrong?<br>
>>>><br>
>>>> Jim<br>
>>>> From: general-bounces@lists.openfabrics.org<br>
>>>> [mailto:general-bounces@lists.openfabrics.org] On Behalf
Of Yicheng <br>
>>>> <br>
> Jia<br>
> <br>
>>>> Sent: Thursday, July 10, 2008 2:59 PM<br>
>>>> To: general@lists.openfabrics.org<br>
>>>> Subject: [ofa-general] minimum sw components requirement
for <br>
>>>><br>
>>>> <br>
>> driver/opensm<br>
>><br>
>> <br>
>>>> in a single unmanaged switch network<br>
>>>><br>
>>>><br>
>>>> Hi Folks,<br>
>>>><br>
>>>> I have a IB network which consists of only a single unmanaged
switch, <br>
>>>><br>
>>>> <br>
>> all<br>
>><br>
>> <br>
>>>> end nodes connecting with the switch only need to do RDMA
read/write<br>
>>>> operation with each other. My question is, what are the
indispensable<br>
>>>> modules in driver's core and opensm that make the network
up and run?<br>
>>>><br>
>>>> I've been using only ib_mad module in driver's core with
a managed <br>
>>>><br>
>>>> <br>
>> switch<br>
>><br>
>> <br>
>>>> before, and the network works fine. So I assume that only
the ib_mad<br>
>>>> module<br>
>>>> in driver's core and SM in opensm are mandatory in my
network. The <br>
>>>> <br>
> LIDs<br>
> <br>
>>>> are<br>
>>>> assigned by them. The SA and CM modules are not useful
in my case. Am <br>
>>>> <br>
> I<br>
> <br>
>>>> right?<br>
>>>><br>
>>>> I need to minimize driver and opensm to fit them in my
network, the <br>
>>>> <br>
> HCA<br>
> <br>
>>>> driver is mthca.<br>
>>>><br>
>>>> Best,<br>
>>>> Yicheng<br>
>>>><br>
>>>><br>
>>>><br>
>>>> <br>
> _____________________________________________________________________________<br>
> <br>
>>>> Scanned by IBM Email Security Management Services powered
by <br>
>>>><br>
>>>> <br>
>> MessageLabs.<br>
>><br>
>> <br>
>>>> For more information please visit </font></tt><a href=http://www.ers.ibm.com/><tt><font size=2>http://www.ers.ibm.com<br>
>>>><br>
>>>><br>
>>>><br>
>>>> <br>
> _____________________________________________________________________________<br>
> <br>
>>>> <br>
> _____________________________________________________________________________<br>
> <br>
>>>> Scanned by IBM Email Security Management Services powered
by <br>
>>>><br>
>>>> <br>
>> MessageLabs.<br>
>><br>
>> <br>
>>>> For more information please visit </font></tt><a href=http://www.ers.ibm.com/><tt><font size=2>http://www.ers.ibm.com<br>
>>>><br>
>>>><br>
>>>><br>
>>>> <br>
> _____________________________________________________________________________<br>
> <br>
>>>> <br>
> _____________________________________________________________________________<br>
> <br>
>>>> Scanned by IBM Email Security Management Services powered
by <br>
>>>><br>
>>>> <br>
>> MessageLabs.<br>
>><br>
>> <br>
>>>> For more information please visit </font></tt><a href=http://www.ers.ibm.com/><tt><font size=2>http://www.ers.ibm.com<br>
>>>><br>
>>>><br>
>>>><br>
>>>> <br>
> _____________________________________________________________________________<br>
> <br>
>>>> _______________________________________________<br>
>>>> general mailing list<br>
>>>> general@lists.openfabrics.org<br>
>>>> </font></tt><a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general"><tt><font size=2>http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general<br>
>>>><br>
>>>> To unsubscribe, please visit<br>
>>>> </font></tt><a href="http://openib.org/mailman/listinfo/openib-general"><tt><font size=2>http://openib.org/mailman/listinfo/openib-general<br>
>>>><br>
>>>><br>
>>>> <br>
> _____________________________________________________________________________<br>
> <br>
>>> Scanned by IBM Email Security Management Services powered
by <br>
>>><br>
>>> <br>
>> MessageLabs.<br>
>><br>
>> <br>
>>> For more information please visit </font></tt><a href=http://www.ers.ibm.com/><tt><font size=2>http://www.ers.ibm.com<br>
>>><br>
>>><br>
>>> <br>
> _____________________________________________________________________________<br>
> <br>
>>> <br>
> _____________________________________________________________________________<br>
> <br>
>>> Scanned by IBM Email Security Management Services powered
by <br>
>>><br>
>>> <br>
>> MessageLabs.<br>
>><br>
>> <br>
>>> For more information please visit </font></tt><a href=http://www.ers.ibm.com/><tt><font size=2>http://www.ers.ibm.com<br>
>>><br>
>>><br>
>>> <br>
> _____________________________________________________________________________<br>
> <br>
>>> _______________________________________________<br>
>>> general mailing list<br>
>>> general@lists.openfabrics.org<br>
>>> </font></tt><a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general"><tt><font size=2>http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general<br>
>>><br>
>>> To unsubscribe, please visit<br>
>>> </font></tt><a href="http://openib.org/mailman/listinfo/openib-general"><tt><font size=2>http://openib.org/mailman/listinfo/openib-general<br>
>>><br>
>>><br>
>>> <br>
>> <br>
> _____________________________________________________________________________<br>
> <br>
>> Scanned by IBM Email Security Management Services powered by <br>
>> <br>
> MessageLabs. <br>
> <br>
>> For more information please visit </font></tt><a href=http://www.ers.ibm.com/><tt><font size=2>http://www.ers.ibm.com<br>
>><br>
>> <br>
> _____________________________________________________________________________<br>
> <br>
>><br>
>><br>
>> <br>
> _____________________________________________________________________________<br>
> <br>
>> Scanned by IBM Email Security Management Services powered by <br>
>> <br>
> MessageLabs. For more information please visit </font></tt><a href=http://www.ers.ibm.com/><tt><font size=2>http://www.ers.ibm.com<br>
> <br>
> _____________________________________________________________________________<br>
> <br>
>> ------------------------------------------------------------------------<br>
>><br>
>> _______________________________________________<br>
>> general mailing list<br>
>> general@lists.openfabrics.org<br>
>> </font></tt><a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general"><tt><font size=2>http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general<br>
>><br>
>> To unsubscribe, please visit <br>
>> <br>
> </font></tt><a href="http://openib.org/mailman/listinfo/openib-general"><tt><font size=2>http://openib.org/mailman/listinfo/openib-general<br>
><br>
><br>
> _____________________________________________________________________________<br>
> Scanned by IBM Email Security Management Services powered by MessageLabs.
<br>
> For more information please visit </font></tt><a href=http://www.ers.ibm.com/><tt><font size=2>http://www.ers.ibm.com<br>
> _____________________________________________________________________________<br>
><br>
><br>
><br>
> _____________________________________________________________________________<br>
> Scanned by IBM Email Security Management Services powered by MessageLabs.
For more information please visit </font></tt><a href=http://www.ers.ibm.com/><tt><font size=2>http://www.ers.ibm.com<br>
> _____________________________________________________________________________<br>
> <br>
<br>
<br>
_____________________________________________________________________________<br>
Scanned by IBM Email Security Management Services powered by MessageLabs.
For more information please visit </font></tt><a href=http://www.ers.ibm.com/><tt><font size=2>http://www.ers.ibm.com<br>
_____________________________________________________________________________<br>
</font></tt></a></a></a></a></a></a></a></a></a></a></a></a></a></a></a></a>
<br>
<BR>
_____________________________________________________________________________<BR>
Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com<BR>
_____________________________________________________________________________<BR>