[ofa-general] HCA takes longer time to get LID after opensm restart

[Yicheng Jia]

Hi Hal,

I use opensm with a single unmanaged switch to connect several HCAs. I 
found that HCAs take much longer time to get LID from SA after opensm 
restart without hard reset the switch. If the switch is hard reset before 
opensm restart, then HCAs get their LIDs sooner. I'm wondering if there's 
minhop table pre-existing in the switch which will prevent HCAs from 
regain their LIDs soon. Is there any option to reset the switch during 
opensm start?

Thanks!
Yicheng




"Hal Rosenstock" <hal.rosenstock at gmail.com> 
07/10/2008 09:15 PM

To
"Yicheng Jia" <YJia at tmriusa.com>
cc
"Jim Mott" <jim at mellanox.com>, general at lists.openfabrics.org
Subject
Re: [ofa-general] minimum sw components requirement for driver/opensm in a 
single unmanaged switch network






On Thu, Jul 10, 2008 at 7:39 PM, Yicheng Jia <YJia at tmriusa.com> wrote:
>
>> If you want to avoid all the SM stuff, and are willing to program the
>> switches directly (a few mads)
>
> Is it done by opensm?

Yes.

> What information should be set up in the switch by
> opensm?

Things like the PortInfos and LFT. See IBA spec vol 1 14.2.5

>> Then to figure out QP connections, you just use a function of 3
>> parameters:
>>  my_qp_num = fn_sqp(my_node, target_node, qp_num)
>>  target_qp_num = fn_tqp(my_node, target_node, qp_num)
>> Where qp_num is a small number between 0 and the maximum number of QPs 
you
>> need active between any 2 endpoints.
>
> Can the qp_num be manually assigned?
> Does it need opensm be involved?

SM has nothing to do with QP numbers.

>> If it works, you are done.  If not, reset, up, wait for him to connect 
and
>> send something to you.
>
> Is it reliable? I mean the QPs connection will keep alive during the QPs
> lifecycle?

For one thing, SM needs to try to keep ports at active.

-- Hal

> Best,
> Yicheng
>
>
>
> "Jim Mott" <jim at mellanox.com>
>
> 07/10/2008 04:17 PM
>
> To
> "Yicheng Jia" <YJia at tmriusa.com>, <general at lists.openfabrics.org>
> cc
> Subject
> RE: [ofa-general] minimum sw components requirement for driver/opensm in 
a
> single unmanaged switch network
>
>
>
>
> If you want to avoid all the SM stuff, and are willing to program the
> switches directly (a few mads), then I've used schemes like:
>
> Node LID=base + (switch port * constant) (base=0, constant = 1 works)
>
> Then to figure out QP connections, you just use a function of 3 
parameters:
>   my_qp_num = fn_sqp(my_node, target_node, qp_num)
>   target_qp_num = fn_tqp(my_node, target_node, qp_num)
> Where qp_num is a small number between 0 and the maximum number of QPs 
you
> need active between any 2 endpoints.
>
> With the above scheme, you know your node_id (switch port number), your 
lid,
> the lid of the target node,  and the QPs on both sides.  From there on, 
it
> is clear sailing.  You don't even need to send MADs; just transition the 
QP
> up and try and use it.  If it works, you are done.  If not, reset, up, 
wait
> for him to connect and send something to you.  A little timer to make 
sure
> everybody retries once in awhile and what can go wrong?
>
> Jim
> From: general-bounces at lists.openfabrics.org
> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Yicheng Jia
> Sent: Thursday, July 10, 2008 2:59 PM
> To: general at lists.openfabrics.org
> Subject: [ofa-general] minimum sw components requirement for 
driver/opensm
> in a single unmanaged switch network
>
>
> Hi Folks,
>
> I have a IB network which consists of only a single unmanaged switch, 
all
> end nodes connecting with the switch only need to do RDMA read/write
> operation with each other. My question is, what are the indispensable
> modules in driver's core and opensm that make the network up and run?
>
> I've been using only ib_mad module in driver's core with a managed 
switch
> before, and the network works fine. So I assume that only the ib_mad 
module
> in driver's core and SM in opensm are mandatory in my network. The LIDs 
are
> assigned by them. The SA and CM modules are not useful in my case. Am I
> right?
>
> I need to minimize driver and opensm to fit them in my network, the HCA
> driver is mthca.
>
> Best,
> Yicheng
> 
_____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by 
MessageLabs.
> For more information please visit http://www.ers.ibm.com
> 
_____________________________________________________________________________
>
> 
_____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by 
MessageLabs.
> For more information please visit http://www.ers.ibm.com
> 
_____________________________________________________________________________
>
> 
_____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by 
MessageLabs.
> For more information please visit http://www.ers.ibm.com
> 
_____________________________________________________________________________
>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit
> http://openib.org/mailman/listinfo/openib-general
>

_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs. 
For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________



_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080926/301aa229/attachment.html>