SPAM Re: [ofw] Expose a vendor defined device in ibbus?

Thu Jan 29 05:34:39 PST 2009

Having an application, configuring IOC fabric means that you can't
reveal and work with remote disks at boot time.
Having a configuration file means additional maintance/deployment work.
Changing of the may require fixing of the node images.
What "failover" are you expecting from virtual adapters, running over a
physical one ?
If the latter gets stuck, all of them are dead. :(
[I'm sure missing here something. Could you elaborate more on usage of
these virtual adapters ? TIA]

BTW, if you are going to create IOC devices (from application but) on
base of ibiou.sys discovery, why wouldn't you do the same without
application ?
Let ibiou.sys creates IOC per HCA for every discovered device...

________________________________

	From: mailmeatdkg at gmail.com [mailto:mailmeatdkg at gmail.com] On
Behalf Of Deepak Gupta
	Sent: Thursday, January 29, 2009 2:47 PM
	To: Leonid Keller
	Cc: ofw at lists.openfabrics.org
	Subject: Re: ***SPAM*** Re: [ofw] Expose a vendor defined device
in ibbus?

	Leonid,

	I understand you might be busy with some other stuff too.

	As I already said, this was just a first draft patch and there
are no intentions to move it into the svn repository.
	I am sorry, I guess I misled you by asking the question "whether
above patch is acceptable".

	This patch was just to show a use case of having multiple child
devices associated with a IOC instead of having a single
	child PDO for each IOC.

	Plan is to provide a IOCTL interface in ibiou for creation of
child devices and all device information will be passed through device
	IOCTLs (and not INF file, in this first draf patch I used INF
file for just POC). So no one has to know the IOC parameters at the time
of installation.

	It indeed raise the need of a user application which can scan
the fabric and provide list the of reachable IOCs (particularly Port to
IOC path, like ibsrpdm and ib_qlgc_vnic_query tools on linux do). 
	I am thinking of providing a IOCTL in ibiou to return the list
of reachable IOCs.
	Having a list of reachable IOCs with the user, now user can
write a configuration file describing the child devices (or a GUI to
enter the child device descriptions). This user application will parse
the configuration file and will create the device interfaces.
	Also it will record the all these descriptions into registry so
that from the next boot, ibiou will create the devices without the
intervention of user app.

	>>    - having configured one node, one has to repeate all this
work with all the others or to prepare and clone the image;

	Once a node has been configured with a config file, that can be
used on other nodes too.

	>>    - if a new IOC has been added, one has to go through all
the nodes and to change the Registries. It may also require reimaging of
all the nodes;

	If a new IOC has been added, then yes he would need to make
entries into config file and then user app will take care of rest.

	>>    - what will happen during the WINOF package upgrade ? We
don't want user to repeat all this configuration work, do we ?

	We will have to provide a versioning on above mentioned device
IOCTLs interface of ibiou. Minimal release changes will not affect the
config file structure and hence from a user perspective reconfiguration
will not be required.

	This User Application will help create multiple child devices
for ULP's (SRP/VNIC) and hence much more failover configurations for the
user.

	My basic question is if we agree to above then I can move ahead
for implmenting above mentioned device IOCTLs in ibiou
	and also start doing work on the developing the user application
to do this.

	Please let me know if you have any further queries on it.

	Regards
	Deepak

	On Thu, Jan 29, 2009 at 3:27 PM, Leonid Keller
<leonid at mellanox.co.il> wrote:

		Hi Deepak,

		Sorry for that late reaction. Could not find time
beforehand to look into it... :(

		I'm not sure, whether the advantages of your patch are
worth the deployment problems that it can cause (IMO).
		Here are several ones at the first look:
		    - one has to know all the IOC parameters at the time
of installation;

		        And what if they are yet unknown (IOCs still not
purchased) or unknown to the guy (he doesn't know what IOC at all is) ?
:(
		    - one has to enter IOC parameters during the
installation in some way;
		        Changing INF file is not a convenient way. One
has to provide some GUI to edit the parameters during- and the Registry
after the installation;
		    - having configured one node, one has to repeate all
this work with all the others or to prepare and clone the image;
		    - if a new IOC has been added, one has to go through
all the nodes and to change the Registries. It may also require
reimaging of all the nodes;
		    - what will happen during the WINOF package upgrade
? We don't want user to repeat all this configuration work, do we ?

		What do you think ?

________________________________

			From: ofw-bounces at lists.openfabrics.org
[mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Deepak Gupta
			Sent: Tuesday, January 27, 2009 5:19 PM
			To: ofw at lists.openfabrics.org
			Subject: ***SPAM*** Re: [ofw] Expose a vendor
defined device in ibbus?

			Since I didn't receive any concrete response
from the list about the patch, I still have following questions
unanswered:--

			1) I would like to know who is current owner of
ibiou and get a feedback on whether above patch is acceptable.
			Or Do you suggest some modifications on it ?

			2) If we are ok with this patch then what time
frame should we set for it. Should we target it for coming 2.1 release
or for post 2.1 releases?

			3) Also, I want to move ahead for implementing
IOCTL layer in ibiou for creation of child devices and IOC listings on
user request. Is it acceptable to implement this IOCTL layer for device
creations and IOC listings in ibiou and you suggest to achieve the same
through some other means?

			I want to reiterate the purpose of this patch:--

			This patch is for creating multiple PDO's for a
HCA to IOC path as configured by the user in INF file.
			It will help in having multiple sessions with
the IOC and hence will give some user configurations for fail over.

			Behavior change from current ibiou
implementation:--

			Currently ibiou simply discovers the IOCs and
creates child devices for the IOCs discovered.
			This patch will create multiple child devices
for HCA to IOC paths as configured by the user in INF file.
			If there are no child device configurations for
a particular IOC then no devices will be created for that IOC and hence
will not appear in device manager's device tree.

			This is just a first draft patch to show the
usability and we have plans to implement IOCTLs in ibiou for child
device creations and for listing the reachable IOC PATHs so that a user
application can be written which can list the IOCs and local HCAs to the
user and user can send IOCTLs to create devices for the ULPs.

			Regards
			Deepak

			On Tue, Jan 27, 2009 at 7:09 PM, Deepak Gupta
<deepak.gupta at qlogic.com> wrote:

				Forgot to include the list.

				Regards
				Deepak

				---------- Forwarded message ----------
				From: Deepak Gupta
<deepak.gupta at qlogic.com>
				Date: Sat, Jan 24, 2009 at 5:07 PM
				Subject: Re: [ofw] Expose a vendor
defined device in ibbus?

				To: Fab Tillier
<ftillier at windows.microsoft.com>

				Fab, Nice to see your response.
				Please see below.

				On Fri, Jan 23, 2009 at 11:09 PM, Fab
Tillier <ftillier at windows.microsoft.com> wrote:

				I don't have time to look at this in
depth, but I can tell you from past experience that having a single PDO
for an IOC leads to trouble if you ever have multiple HCAs in the
system.  If you have a miniport driver (like NDIS or StorPort), the port
driver takes care of DMA mappings of user-provided buffers.  That DMA
mapping goes down to the PCI driver for the particular HCA, so you could
potentially have a mapping that isn't valid for one of the multiple HCAs
in the system.  It's best IMO to have a PDO per IOC per HCA (this allows
automatic path migration to work in multi-port HCAs), and push failover
between IOCs to a higher level (LBFO/MPIO).

				I had the same concern while
implementing the patch.
				So I tried to dig into the stack till
h/w drivers and it came to me that all DMA mappings are being
				handled by PCI driver.
				So this patch creates PDO's per IOC per
HCA. 
				User can specify the child device
descriptions in INF file (later on we can think of having a IOCTL
interface in ibiou) and HCA to IOC path.
				ibiou will create child PDO's only if
HCA to IOC path is present.

				Any more suggestions are welcome!!!

				Regards
				Deepak

				-Fab

				From: mailmeatdkg at gmail.com
[mailto:mailmeatdkg at gmail.com] On Behalf Of Deepak Gupta
				Sent: Friday, January 23, 2009 6:41 AM
				To: James Yang
				Cc: Leonid Keller; Fab Tillier;
ofw at lists.openfabrics.org 

				Subject: Re: [ofw] Expose a vendor
defined device in ibbus?

				Did any one get the chance to have a
look at the patch?

				I am really concerned with the tight
coupling of representing each single IOC as a single PDO.

				Regards
				Deepak

				On Wed, Jan 21, 2009 at 5:55 PM, Deepak
Gupta <deepak.gupta at qlogic.com> wrote:

				All,

				Based on James patch of creating user
defined devices tied to local HCA ports, I have created a first draft
patch attached herewith 
				to allow "ibiou" to create user defined
devices which will associate with IOC paths.

				This patch does the following:-

				Now ibiou will not create child devices
on it's own when it discovers a IOC.
				Instead it will create child devices
only when it is instructed to do so.
				For the sake of this patch, currently it
uses name/description of the devices hard coded into INF file of ibiou
driver.
				Later on we can implement a IOCTL to
pass this information to the driver (more below)

				It will make a listing of devices to be
created in it's DriverEntry routine.
				When it will get IOC_PNP_ADD pnp events,
it will check in device list prepare
				earlier. If it finds device(s) in it's
list which corresponds to the new IOC
				discovered then it will create those
child devices and a similar mechanism
				will happen when ibiou will get
IOC_PNP_ADD event.

				This more of a initial patch for giving
an insight into what we want to achieve.
				On a longer run we are thinking of
providing a IOCTL support in ibiou driver for creating child devices
(like vnic/srp/etc) and
				differnet vendor specific applications
can create those devices and have their driver loaded on top of that.

				Following is the advantage of this
functionality:-

				   Earlier there was only one device
created per IOC on host side.
				   Now there can be multiple devices on
host side per IOC as configured in the registry.
				   It will be many to one function i.e
there can be many devices on host side which will target to the same
IOC.
				   It will be like having different
sessions with the same IOC.

				Please have a look at it and let me know
of your comments/suggestions/feeback.

				NOTE--> Device descriptions in the INF
file of this patch creates devices that corresponds to a IOC PATH
(CAGUID and IOCGUID).
				        Those values are local to my
machine and you should change it according to your fabric.

				Regards
				Deepak

				On Tue, Jan 20, 2009 at 1:37 AM, James
Yang <jyang at xsigo.com> wrote:

				The proposal is to use registry key to
define vendor devices, and the registry key is global to the driver. The
assumption is that multiple HCA cards will have the same vendor defined
devices. It cannot support one HCA with vendor-A device, and the other
HCA with vendor-B device, in the same system. At default all hcas will
have IPoIB as child devices.

				Thanks,

				James

________________________________

				From: Leonid Keller
[mailto:leonid at mellanox.co.il] 
				Sent: Sunday, January 18, 2009 5:57 AM
				To: James Yang; Deepak Gupta; Fab
Tillier

				Cc: ofw at lists.openfabrics.org
				Subject: RE: [ofw] Expose a vendor
defined device in ibbus?

				After first look: why did you put the
list of the created devices into Globals (and not, say, FDO) ?

				How it will work for multi-home machine
? (several HCA cards)

				Would anybode like to check it for
various partition keys ?

________________________________

				From: ofw-bounces at lists.openfabrics.org
[mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of James Yang
				Sent: Tuesday, January 06, 2009 10:11 PM
				To: Deepak Gupta; Fab Tillier
				Cc: ofw at lists.openfabrics.org
				Subject: RE: [ofw] Expose a vendor
defined device in ibbus?

				Hi,

				Please review the patch to create user
defined devices by reading from registry. By default there is only one
IpoIB device enabled in mlx4_hca.inx file. This patch will only work for
ConnectX.

				The paritition key if set to default to
FFFF, I didn't test on other value. And the Ioctl part to add partition
key may also need to be verified.

				Thanks,

				James

________________________________

				From: mailmeatdkg at gmail.com
[mailto:mailmeatdkg at gmail.com] On Behalf Of Deepak Gupta
				Sent: Monday, January 05, 2009 10:51 PM
				To: Fab Tillier
				Cc: Tzachi Dar; James Yang;
ofw at lists.openfabrics.org
				Subject: Re: [ofw] Expose a vendor
defined device in ibbus?

				Have a gr8 New Year to all the
members!!!.

				Do you we any updates on "vendor defined
device in ibbus"?

				I wanted to create multiple vnic
interfaces irrespective of number of reachable IOCs.

				Currently I am creating vnic child
devices on "root" bus.
				Inside VNIC driver, I am looking for
arrival GUID_IB_AL_INTERFACE and hence contacting the IBAL.
				But since vnic devices are root
enumerated, VNIC driver gets loaded very earlier in boot phase (Before
"Extended Base" group to which IB Stack drivers belong) and hence VNIC
device interfaces are not getting initialized properly.

				If we are providing a vendor defined
device functionality in ibbus in near future, then it would be worth for
me to wait for it.

				Can any one please comment on this.

				Regards
				Deepak

				On Mon, Dec 22, 2008 at 1:12 PM, Deepak
Gupta <deepak.gupta at qlogic.com> wrote:

				All,

				I came across one more question in my
mind which are I think is not clear to me after reading the whole
thread.

				In new design being discussed:- 
				Are we making sure that we can have more
than one child devices configured for the same IOC.
				Currently, there is one child device
created per IOC discovered.

				Having more than one child device
configured for same IOC is a requirement if a user wants two different
ULP interfaces to be created on host side.
				Consider a case in which a host is
connected to a single IOC and IOC is connected to a ethernet network via
switch.
				If there are two different IP subnets
then there is a requirement of two different Ethernet interfaces on the
host side too.

				Please let me know if you need more
clarification of my question.

				Regards
				Deepak

				On Sat, Dec 20, 2008 at 2:19 PM, Deepak
Gupta <deepak.gupta at qlogic.com> wrote:

				Please see below.

				Regards
				Deepak

				On Sat, Dec 20, 2008 at 12:42 AM, Fab
Tillier <ftillier at windows.microsoft.com> wrote:

				> On Wed, Dec 17, 2008 at 11:43 PM, Fab
Tillier
				> <ftillier at windows.microsoft.com>
wrote:

				>> Are there other properties that I
have missed that are needed?
				>
				> We need a way in which devices created
should be configured for
				> failovers ( ULPs like VNIC, SRP need
more configurable failovers).
				> Looking at IBAL's code it create the
devices based on the reachable
				> IOC's and thus failover's are possible
across the HCA/ports and not
				> across two different IOCs.
				> Users can have a case in which two
different IOCs connected to same
				> physical network/storage (redundancy
is provided for high availability)
				> and want a failover across the IOCs.

				This would be done via LBFO for network
devices, and MPIO for storage devices.  I think having the bus driver
report a single IOC that really maps to two IOCs on the fabric is asking
for management problems.  Further, LBFO/MPIO can provide failover
between different device types, so the failover devices don't have to be
identical HW.

				I don't know about how MPIO works. But
for LBFO,  BundleID param will have to be included in extended params
then so that user gets the freedom of bundling  different failover
configurations.

				-Fab

_______________________________________________
				ofw mailing list
				ofw at lists.openfabrics.org

http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw

_______________________________________________
				ofw mailing list
				ofw at lists.openfabrics.org

http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw

		_______________________________________________
		ofw mailing list
		ofw at lists.openfabrics.org

http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofw/attachments/20090129/94b24f2e/attachment.html>

***SPAM*** Re: [ofw] Expose a vendor defined device in ibbus?

SPAM Re: [ofw] Expose a vendor defined device in ibbus?