[ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network

[Yicheng Jia]

My operation is quite simple: connect QPs and do RDMA read/write. In this 
case, the opensm is not in need when the subnet is up, correct?

Thanks!
Yicheng




"Dotan Barak" <dotanba at gmail.com> 
08/21/2008 02:33 PM

To
"Yicheng Jia" <YJia at tmriusa.com>
cc
"Hal Rosenstock" <hal.rosenstock at gmail.com>, general at lists.openfabrics.org
Subject
Re: [ofa-general] minimum sw components requirement for driver/opensm in a 
single unmanaged switch network






On Thu, Aug 21, 2008 at 10:16 PM, Yicheng Jia <YJia at tmriusa.com> wrote:
>
> Hi Hal,
>
> Can opensm just run once? When the subnet is up, it can exit assume that 
no
> change will be made in the subnet.
>
Yes, depend on the serives that you will need/use.

For example: if you use operations that requires SA query, you must
have a live SM.

If you will connect the QPs in the subnet by yourself (for example,
using socket) you can manage without a live SM in the subnet ...

Dotan
> Thanks!
> Yicheng
>
>
>
> "Hal Rosenstock" <hal.rosenstock at gmail.com>
>
> 07/10/2008 09:15 PM
>
> To
> "Yicheng Jia" <YJia at tmriusa.com>
> cc
> "Jim Mott" <jim at mellanox.com>, general at lists.openfabrics.org
> Subject
> Re: [ofa-general] minimum sw components requirement for driver/opensm in 
a
> single unmanaged switch network
>
>
>
>
> On Thu, Jul 10, 2008 at 7:39 PM, Yicheng Jia <YJia at tmriusa.com> wrote:
>>
>>> If you want to avoid all the SM stuff, and are willing to program the
>>> switches directly (a few mads)
>>
>> Is it done by opensm?
>
> Yes.
>
>> What information should be set up in the switch by
>> opensm?
>
> Things like the PortInfos and LFT. See IBA spec vol 1 14.2.5
>
>>> Then to figure out QP connections, you just use a function of 3
>>> parameters:
>>>  my_qp_num = fn_sqp(my_node, target_node, qp_num)
>>>  target_qp_num = fn_tqp(my_node, target_node, qp_num)
>>> Where qp_num is a small number between 0 and the maximum number of QPs
>>> you
>>> need active between any 2 endpoints.
>>
>> Can the qp_num be manually assigned?
>> Does it need opensm be involved?
>
> SM has nothing to do with QP numbers.
>
>>> If it works, you are done.  If not, reset, up, wait for him to connect
>>> and
>>> send something to you.
>>
>> Is it reliable? I mean the QPs connection will keep alive during the 
QPs
>> lifecycle?
>
> For one thing, SM needs to try to keep ports at active.
>
> -- Hal
>
>> Best,
>> Yicheng
>>
>>
>>
>> "Jim Mott" <jim at mellanox.com>
>>
>> 07/10/2008 04:17 PM
>>
>> To
>> "Yicheng Jia" <YJia at tmriusa.com>, <general at lists.openfabrics.org>
>> cc
>> Subject
>> RE: [ofa-general] minimum sw components requirement for driver/opensm 
in a
>> single unmanaged switch network
>>
>>
>>
>>
>> If you want to avoid all the SM stuff, and are willing to program the
>> switches directly (a few mads), then I've used schemes like:
>>
>> Node LID=base + (switch port * constant) (base=0, constant = 1 works)
>>
>> Then to figure out QP connections, you just use a function of 3
>> parameters:
>>   my_qp_num = fn_sqp(my_node, target_node, qp_num)
>>   target_qp_num = fn_tqp(my_node, target_node, qp_num)
>> Where qp_num is a small number between 0 and the maximum number of QPs 
you
>> need active between any 2 endpoints.
>>
>> With the above scheme, you know your node_id (switch port number), your
>> lid,
>> the lid of the target node,  and the QPs on both sides.  From there on, 
it
>> is clear sailing.  You don't even need to send MADs; just transition 
the
>> QP
>> up and try and use it.  If it works, you are done.  If not, reset, up,
>> wait
>> for him to connect and send something to you.  A little timer to make 
sure
>> everybody retries once in awhile and what can go wrong?
>>
>> Jim
>> From: general-bounces at lists.openfabrics.org
>> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Yicheng Jia
>> Sent: Thursday, July 10, 2008 2:59 PM
>> To: general at lists.openfabrics.org
>> Subject: [ofa-general] minimum sw components requirement for 
driver/opensm
>> in a single unmanaged switch network
>>
>>
>> Hi Folks,
>>
>> I have a IB network which consists of only a single unmanaged switch, 
all
>> end nodes connecting with the switch only need to do RDMA read/write
>> operation with each other. My question is, what are the indispensable
>> modules in driver's core and opensm that make the network up and run?
>>
>> I've been using only ib_mad module in driver's core with a managed 
switch
>> before, and the network works fine. So I assume that only the ib_mad
>> module
>> in driver's core and SM in opensm are mandatory in my network. The LIDs
>> are
>> assigned by them. The SA and CM modules are not useful in my case. Am I
>> right?
>>
>> I need to minimize driver and opensm to fit them in my network, the HCA
>> driver is mthca.
>>
>> Best,
>> Yicheng
>>
>> 
_____________________________________________________________________________
>> Scanned by IBM Email Security Management Services powered by 
MessageLabs.
>> For more information please visit http://www.ers.ibm.com
>>
>> 
_____________________________________________________________________________
>>
>>
>> 
_____________________________________________________________________________
>> Scanned by IBM Email Security Management Services powered by 
MessageLabs.
>> For more information please visit http://www.ers.ibm.com
>>
>> 
_____________________________________________________________________________
>>
>>
>> 
_____________________________________________________________________________
>> Scanned by IBM Email Security Management Services powered by 
MessageLabs.
>> For more information please visit http://www.ers.ibm.com
>>
>> 
_____________________________________________________________________________
>>
>> _______________________________________________
>> general mailing list
>> general at lists.openfabrics.org
>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>
>> To unsubscribe, please visit
>> http://openib.org/mailman/listinfo/openib-general
>>
>
> 
_____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by 
MessageLabs.
> For more information please visit http://www.ers.ibm.com
> 
_____________________________________________________________________________
>
>
> 
_____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by 
MessageLabs.
> For more information please visit http://www.ers.ibm.com
> 
_____________________________________________________________________________
>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit
> http://openib.org/mailman/listinfo/openib-general
>

_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs. 
For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________



_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080821/4cafaab4/attachment.html>