[ofw] Doing queries on subnet every 30 seconds

[Smith, Stan]

Hello,
  A couple of points:

1) mthca.inx needs the IocPollInterval change also.

2) To clarify the handling of IOC/IOU (say SRP for instance) in light of your proposed changes.
With this change the default behavior is that no IOC/IOU will be recognized when integrated into the fabric.
In order for an IOC/IOU to be recognized, all fabric nodes which desire to recognize an IOC/IOU must have their registry entry ' IocPollInterval ' reset to 30000 and then rebooted.

On order to be more a wee bit more dynamic, could one add to the function __ioc_pnp_cb() in file al_ioc_pnp.c the following port IOC PNP events?

IB_PNP_IOU_ADD
IB_PNP_IOU_REMOVE
IB_PNP_IOC_ADD
IB_PNP_IOC_REMOVE
IB_PNP_IOC_PATH_ADD
IB_PNP_IOC_PATH_REMOVE

Each event would start an ioc sweep if one was not already in progress?
An IOC sweep, such that __ioc_pnp_timer_cb( p_ioc_mgr ) is called.

What about Hal's suggestion on reworking the IOC sweep mechanisms to use newer SA routines to lighten the IOC sweep impact?
Is this work scheduled to be included in winOFED 3.0 or near future?

Stan.
  

>-----Original Message-----
>From: ofw-bounces at lists.openfabrics.org [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Tzachi Dar
>Sent: Tuesday, May 31, 2011 5:06 AM
>To: Tzachi Dar; Hal Rosenstock; Fab Tillier
>Cc: ofw at lists.openfabrics.org
>Subject: Re: [ofw] Doing queries on subnet every 30 seconds
>
>I have changed the default on checkin #3117
>
>Thanks
>Tzachi
>
>Index: B:/users/tzachid/projinf6/trunk/core/al/kernel/al_ioc_pnp.c
>===================================================================
>--- B:/users/tzachid/projinf6/trunk/core/al/kernel/al_ioc_pnp.c	(revision 8124)
>+++ B:/users/tzachid/projinf6/trunk/core/al/kernel/al_ioc_pnp.c	(revision 8125)
>@@ -294,7 +294,6 @@
> 	cl_async_proc_item_t	async_item;
> 	sweep_state_t			state;
> 	ioc_pnp_svc_t			*p_svc;
>-	atomic32_t				query_cnt;
> 	cl_fmap_t				iou_map;
>
> }	ioc_sweep_results_t;
>@@ -313,7 +312,7 @@
> ioc_pnp_mgr_t	*gp_ioc_pnp = NULL;
> uint32_t		g_ioc_query_timeout = 250;
> uint32_t		g_ioc_query_retries = 4;
>-uint32_t		g_ioc_poll_interval = 30000;
>+uint32_t		g_ioc_poll_interval = 0;
>
>
>
>Index: B:/users/tzachid/projinf6/trunk/hw/mlx4/kernel/hca/mlx4_hca.inx
>===================================================================
>--- B:/users/tzachid/projinf6/trunk/hw/mlx4/kernel/hca/mlx4_hca.inx	(revision 8124)
>+++ B:/users/tzachid/projinf6/trunk/hw/mlx4/kernel/hca/mlx4_hca.inx	(revision 8125)
>@@ -411,7 +411,7 @@
> HKR,"Parameters","SmiPollInterval",%REG_DWORD_NO_CLOBBER%,20000
> HKR,"Parameters","IocQueryTimeout",%REG_DWORD_NO_CLOBBER%,250
> HKR,"Parameters","IocQueryRetries",%REG_DWORD_NO_CLOBBER%,4
>-HKR,"Parameters","IocPollInterval",%REG_DWORD_NO_CLOBBER%,30000
>+HKR,"Parameters","IocPollInterval",%REG_DWORD_NO_CLOBBER%,0
> HKR,"Parameters","DebugFlags",%REG_DWORD%,0x80000000
> HKR,"Parameters","ReportPortNIC",%REG_DWORD%,1
>
>
>
>> -----Original Message-----
>> From: ofw-bounces at lists.openfabrics.org [mailto:ofw-
>> bounces at lists.openfabrics.org] On Behalf Of Tzachi Dar
>> Sent: Tuesday, May 03, 2011 9:23 PM
>> To: Hal Rosenstock; Fab Tillier
>> Cc: ofw at lists.openfabrics.org
>> Subject: Re: [ofw] Doing queries on subnet every 30 seconds
>>
>> The new algorithm can be implemented later (if you will tell us exactly what
>> it is).
>>
>> Currently we are only talking about a change in the default.
>>
>> Thanks
>> Tzachi
>>
>> > -----Original Message-----
>> > From: Hal Rosenstock [mailto:hal.rosenstock at gmail.com]
>> > Sent: Tuesday, May 03, 2011 9:20 PM
>> > To: Fab Tillier
>> > Cc: Tzachi Dar; ofw at lists.openfabrics.org
>> > Subject: Re: [ofw] Doing queries on subnet every 30 seconds
>> >
>> > On Tue, May 3, 2011 at 2:15 PM, Fab Tillier <ftillier at microsoft.com> wrote:
>> > > Hal Rosenstock wrote on Mon, 2 May 2011 at 05:09:46
>> > >
>> > >> On Mon, May 2, 2011 at 4:54 AM, Tzachi Dar <tzachid at mellanox.co.il>
>> wrote:
>> > >>> It seems that currently we have code that scans for srp targets every 30
>> > >>> seconds (and creates qpr).
>> > >>>
>> > >>> See the function __ioc_query_sa for more details.
>> > >>>
>> > >>
>> > >> What is __ioc_query_sa doing ? Where is this code found ? Is this an
>> > >> SA query ? Which one ?
>> > >
>> > > Basically, if I recall correctly, it queries the SM for all port records
>> and
>> > all path records, does a join of the tables to get all paths to ports that
>> > have the IOC bit set, and then sends DM MADs to discover I/O devices.
>> >
>> > There is a much better optional algorithm for this with more recent SMs.
>> >
>> > -- Hal
>> >
>> > > -Fab
>> > >
>> > >>
>> > >> -- Hal
>> > >>
>> > >>>
>> > >>> When the cluster gets big it creates load on the sm.
>> > >>>
>> > >>>
>> > >>>
>> > >>> Is there any objection to disable it by default?
>> > >>>
>> > >>>
>> > >>>
>> > >>> Thanks
>> > >>>
>> > >>> Tzachi
>> > >>>
>> > >>> _______________________________________________
>> > >>> ofw mailing list
>> > >>> ofw at lists.openfabrics.org
>> > >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>> > >>>
>> > >> _______________________________________________
>> > >> ofw mailing list
>> > >> ofw at lists.openfabrics.org
>> > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>> > >
>> _______________________________________________
>> ofw mailing list
>> ofw at lists.openfabrics.org
>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>_______________________________________________
>ofw mailing list
>ofw at lists.openfabrics.org
>http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw