[ofw] Doing queries on subnet every 30 seconds

Tue May 31 13:56:48 PDT 2011

Additionally an IOC sweep would be initiated, if not already sweeping, when a IRP_MN_QUERY_DEVICE_RELATIONS is received.
IRP_MN_QUERY_DEVICE_RELATIONS is generated when a 'device rescan' is requested; this would handle the case of client IOU users (SRP client) finding new fabric devices.

>-----Original Message-----
>From: ofw-bounces at lists.openfabrics.org [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Smith, Stan
>Sent: Tuesday, May 31, 2011 1:10 PM
>To: Tzachi Dar; Hal Rosenstock; Fab Tillier
>Cc: ofw at lists.openfabrics.org
>Subject: Re: [ofw] Doing queries on subnet every 30 seconds
>
>Hello,
>  A couple of points:
>
>1) mthca.inx needs the IocPollInterval change also.
>
>2) To clarify the handling of IOC/IOU (say SRP for instance) in light of your proposed changes.
>With this change the default behavior is that no IOC/IOU will be recognized when integrated into the fabric.
>In order for an IOC/IOU to be recognized, all fabric nodes which desire to recognize an IOC/IOU must have their registry entry '
>IocPollInterval ' reset to 30000 and then rebooted.
>
>On order to be more a wee bit more dynamic, could one add to the function __ioc_pnp_cb() in file al_ioc_pnp.c the following port IOC PNP
>events?
>
>IB_PNP_IOU_ADD
>IB_PNP_IOU_REMOVE
>IB_PNP_IOC_ADD
>IB_PNP_IOC_REMOVE
>IB_PNP_IOC_PATH_ADD
>IB_PNP_IOC_PATH_REMOVE
>
>Each event would start an ioc sweep if one was not already in progress?
>An IOC sweep, such that __ioc_pnp_timer_cb( p_ioc_mgr ) is called.
>
>What about Hal's suggestion on reworking the IOC sweep mechanisms to use newer SA routines to lighten the IOC sweep impact?
>Is this work scheduled to be included in winOFED 3.0 or near future?
>
>Stan.
>
>
>>-----Original Message-----
>>From: ofw-bounces at lists.openfabrics.org [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Tzachi Dar
>>Sent: Tuesday, May 31, 2011 5:06 AM
>>To: Tzachi Dar; Hal Rosenstock; Fab Tillier
>>Cc: ofw at lists.openfabrics.org
>>Subject: Re: [ofw] Doing queries on subnet every 30 seconds
>>
>>I have changed the default on checkin #3117
>>
>>Thanks
>>Tzachi
>>
>>Index: B:/users/tzachid/projinf6/trunk/core/al/kernel/al_ioc_pnp.c
>>===================================================================
>>--- B:/users/tzachid/projinf6/trunk/core/al/kernel/al_ioc_pnp.c	(revision 8124)
>>+++ B:/users/tzachid/projinf6/trunk/core/al/kernel/al_ioc_pnp.c	(revision 8125)
>>@@ -294,7 +294,6 @@
>> 	cl_async_proc_item_t	async_item;
>> 	sweep_state_t			state;
>> 	ioc_pnp_svc_t			*p_svc;
>>-	atomic32_t				query_cnt;
>> 	cl_fmap_t				iou_map;
>>
>> }	ioc_sweep_results_t;
>>@@ -313,7 +312,7 @@
>> ioc_pnp_mgr_t	*gp_ioc_pnp = NULL;
>> uint32_t		g_ioc_query_timeout = 250;
>> uint32_t		g_ioc_query_retries = 4;
>>-uint32_t		g_ioc_poll_interval = 30000;
>>+uint32_t		g_ioc_poll_interval = 0;
>>
>>
>>
>>Index: B:/users/tzachid/projinf6/trunk/hw/mlx4/kernel/hca/mlx4_hca.inx
>>===================================================================
>>--- B:/users/tzachid/projinf6/trunk/hw/mlx4/kernel/hca/mlx4_hca.inx	(revision 8124)
>>+++ B:/users/tzachid/projinf6/trunk/hw/mlx4/kernel/hca/mlx4_hca.inx	(revision 8125)
>>@@ -411,7 +411,7 @@
>> HKR,"Parameters","SmiPollInterval",%REG_DWORD_NO_CLOBBER%,20000
>> HKR,"Parameters","IocQueryTimeout",%REG_DWORD_NO_CLOBBER%,250
>> HKR,"Parameters","IocQueryRetries",%REG_DWORD_NO_CLOBBER%,4
>>-HKR,"Parameters","IocPollInterval",%REG_DWORD_NO_CLOBBER%,30000
>>+HKR,"Parameters","IocPollInterval",%REG_DWORD_NO_CLOBBER%,0
>> HKR,"Parameters","DebugFlags",%REG_DWORD%,0x80000000
>> HKR,"Parameters","ReportPortNIC",%REG_DWORD%,1
>>
>>
>>
>>> -----Original Message-----
>>> From: ofw-bounces at lists.openfabrics.org [mailto:ofw-
>>> bounces at lists.openfabrics.org] On Behalf Of Tzachi Dar
>>> Sent: Tuesday, May 03, 2011 9:23 PM
>>> To: Hal Rosenstock; Fab Tillier
>>> Cc: ofw at lists.openfabrics.org
>>> Subject: Re: [ofw] Doing queries on subnet every 30 seconds
>>>
>>> The new algorithm can be implemented later (if you will tell us exactly what
>>> it is).
>>>
>>> Currently we are only talking about a change in the default.
>>>
>>> Thanks
>>> Tzachi
>>>
>>> > -----Original Message-----
>>> > From: Hal Rosenstock [mailto:hal.rosenstock at gmail.com]
>>> > Sent: Tuesday, May 03, 2011 9:20 PM
>>> > To: Fab Tillier
>>> > Cc: Tzachi Dar; ofw at lists.openfabrics.org
>>> > Subject: Re: [ofw] Doing queries on subnet every 30 seconds
>>> >
>>> > On Tue, May 3, 2011 at 2:15 PM, Fab Tillier <ftillier at microsoft.com> wrote:
>>> > > Hal Rosenstock wrote on Mon, 2 May 2011 at 05:09:46
>>> > >
>>> > >> On Mon, May 2, 2011 at 4:54 AM, Tzachi Dar <tzachid at mellanox.co.il>
>>> wrote:
>>> > >>> It seems that currently we have code that scans for srp targets every 30
>>> > >>> seconds (and creates qpr).
>>> > >>>
>>> > >>> See the function __ioc_query_sa for more details.
>>> > >>>
>>> > >>
>>> > >> What is __ioc_query_sa doing ? Where is this code found ? Is this an
>>> > >> SA query ? Which one ?
>>> > >
>>> > > Basically, if I recall correctly, it queries the SM for all port records
>>> and
>>> > all path records, does a join of the tables to get all paths to ports that
>>> > have the IOC bit set, and then sends DM MADs to discover I/O devices.
>>> >
>>> > There is a much better optional algorithm for this with more recent SMs.
>>> >
>>> > -- Hal
>>> >
>>> > > -Fab
>>> > >
>>> > >>
>>> > >> -- Hal
>>> > >>
>>> > >>>
>>> > >>> When the cluster gets big it creates load on the sm.
>>> > >>>
>>> > >>>
>>> > >>>
>>> > >>> Is there any objection to disable it by default?
>>> > >>>
>>> > >>>
>>> > >>>
>>> > >>> Thanks
>>> > >>>
>>> > >>> Tzachi
>>> > >>>
>>> > >>> _______________________________________________
>>> > >>> ofw mailing list
>>> > >>> ofw at lists.openfabrics.org
>>> > >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>>> > >>>
>>> > >> _______________________________________________
>>> > >> ofw mailing list
>>> > >> ofw at lists.openfabrics.org
>>> > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>>> > >
>>> _______________________________________________
>>> ofw mailing list
>>> ofw at lists.openfabrics.org
>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>>_______________________________________________
>>ofw mailing list
>>ofw at lists.openfabrics.org
>>http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>_______________________________________________
>ofw mailing list
>ofw at lists.openfabrics.org
>http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw