Forgot to include the list.<br><br>Regards<br>Deepak<br><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Deepak Gupta</b> <span dir="ltr"><<a href="mailto:deepak.gupta@qlogic.com">deepak.gupta@qlogic.com</a>></span><br>
Date: Sat, Jan 24, 2009 at 5:07 PM<br>Subject: Re: [ofw] Expose a vendor defined device in ibbus?<br>To: Fab Tillier <<a href="mailto:ftillier@windows.microsoft.com">ftillier@windows.microsoft.com</a>><br><br><br>Fab, Nice to see your response.<br>
Please see below.<br><br><div class="gmail_quote"><div class="Ih2E3d">On Fri, Jan 23, 2009 at 11:09 PM, Fab Tillier <span dir="ltr"><<a href="mailto:ftillier@windows.microsoft.com" target="_blank">ftillier@windows.microsoft.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">I don't have time to look at this in depth, but I can tell you from
past experience that having a single PDO for an IOC leads to trouble if you
ever have multiple HCAs in the system. If you have a miniport driver (like
NDIS or StorPort), the port driver takes care of DMA mappings of user-provided
buffers. That DMA mapping goes down to the PCI driver for the particular HCA,
so you could potentially have a mapping that isn't valid for one of the
multiple HCAs in the system. It's best IMO to have a PDO per IOC per HCA (this
allows automatic path migration to work in multi-port HCAs), and push failover
between IOCs to a higher level (LBFO/MPIO).</span></p></div></div></blockquote></div><div><br>I had the same concern while implementing the patch.<br>So I tried to dig into the stack till h/w drivers and it came to me that all DMA mappings are being<br>
handled by PCI driver.<br>So this patch creates PDO's per IOC per HCA. <br>User can specify the child device descriptions in INF file (later on we can think of having a IOCTL interface in ibiou) and HCA to IOC path.<br>
ibiou will create child PDO's only if HCA to IOC path is present.<br><br>Any more suggestions are welcome!!!<br><br>Regards<br><font color="#888888">Deepak<br> <br></font></div><div><div></div><div class="Wj3C7c"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div link="blue" vlink="purple" lang="EN-US"><div><p><span style="font-size: 11pt; color: rgb(31, 73, 125);"></span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">-Fab</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<div style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color blue; border-width: medium medium medium 1.5pt; padding: 0in 0in 0in 4pt;">
<div>
<div style="border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; border-width: 1pt medium medium; padding: 3pt 0in 0in;">
<p><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;">
<a href="mailto:mailmeatdkg@gmail.com" target="_blank">mailmeatdkg@gmail.com</a> [mailto:<a href="mailto:mailmeatdkg@gmail.com" target="_blank">mailmeatdkg@gmail.com</a>] <b>On Behalf Of </b>Deepak
Gupta<br>
<b>Sent:</b> Friday, January 23, 2009 6:41 AM<br>
<b>To:</b> James Yang<br>
<b>Cc:</b> Leonid Keller; Fab Tillier; <a href="mailto:ofw@lists.openfabrics.org" target="_blank">ofw@lists.openfabrics.org</a><div><div></div><div><br>
<b>Subject:</b> Re: [ofw] Expose a vendor defined device in ibbus?</div></div></span></p>
</div>
</div><div><div></div><div>
<p> </p>
<p style="margin-bottom: 12pt;">Did any one get the chance to
have a look at the patch?<br>
<br>
I am really concerned with the tight coupling of representing each single IOC
as a single PDO.<br>
<br>
Regards<br>
Deepak<br>
<br>
<br>
</p>
<div>
<p>On Wed, Jan 21, 2009 at 5:55 PM, Deepak Gupta <<a href="mailto:deepak.gupta@qlogic.com" target="_blank">deepak.gupta@qlogic.com</a>>
wrote:</p>
<p style="margin-bottom: 12pt;">All,<br>
<br>
Based on James patch of creating user defined devices tied to local HCA ports,
I have created a first draft patch attached herewith <br>
to allow "ibiou" to create user defined devices which will associate
with IOC paths.<br>
<br>
This patch does the following:-<br>
<br>
Now ibiou will not create child devices on it's own when it discovers a IOC.<br>
Instead it will create child devices only when it is instructed to do so.<br>
For the sake of this patch, currently it uses name/description of the devices
hard coded into INF file of ibiou driver.<br>
Later on we can implement a IOCTL to pass this information to the driver (more
below)<br>
<br>
It will make a listing of devices to be created in it's DriverEntry routine.<br>
When it will get IOC_PNP_ADD pnp events, it will check in device list prepare<br>
earlier. If it finds device(s) in it's list which corresponds to the new IOC<br>
discovered then it will create those child devices and a similar mechanism<br>
will happen when ibiou will get IOC_PNP_ADD event.<br>
<br>
This more of a initial patch for giving an insight into what we want to
achieve.<br>
On a longer run we are thinking of providing a IOCTL support in ibiou driver
for creating child devices (like vnic/srp/etc) and<br>
differnet vendor specific applications can create those devices and have their
driver loaded on top of that.<br>
<br>
Following is the advantage of this functionality:-<br>
<br>
Earlier there was only one device created per IOC on host side.<br>
Now there can be multiple devices on host side per IOC as
configured in the registry.<br>
It will be many to one function i.e there can be many devices on
host side which will target to the same IOC.<br>
It will be like having different sessions with the same IOC.<br>
<br>
Please have a look at it and let me know of your comments/suggestions/feeback.<br>
<br>
NOTE--> Device descriptions in the INF file of this patch creates devices
that corresponds to a IOC PATH (CAGUID and IOCGUID).<br>
Those values are local to my machine
and you should change it according to your fabric.<br>
<br>
Regards<br>
<span style="color: rgb(136, 136, 136);">Deepak<br>
<br>
<br>
</span></p>
<div>
<div>
<div>
<p>On Tue, Jan 20, 2009 at 1:37 AM, James Yang <<a href="mailto:jyang@xsigo.com" target="_blank">jyang@xsigo.com</a>> wrote:</p>
</div>
</div>
<blockquote style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color rgb(204, 204, 204); border-width: medium medium medium 1pt; padding: 0in 0in 0in 6pt; margin-left: 4.8pt; margin-right: 0in;">
<div>
<div>
<div>
<div>
<p><span style="font-size: 10pt; color: navy;">The
proposal is to use registry key to define vendor devices, and the registry key
is global to the driver. The assumption is that multiple HCA cards will have
the same vendor defined devices. It cannot support one HCA with vendor-A
device, and the other HCA with vendor-B device, in the same system. At default
all hcas will have IPoIB as child devices.</span></p>
<p><span style="font-size: 10pt; color: navy;"> </span></p>
<p><span style="font-size: 10pt; color: navy;">Thanks,</span></p>
<p><span style="font-size: 10pt; color: navy;">James</span></p>
<p><span style="font-size: 10pt; color: navy;"> </span></p>
<div>
<div style="text-align: center;" align="center">
<hr align="center" size="2" width="100%">
</div>
<p><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;"> Leonid Keller
[mailto:<a href="mailto:leonid@mellanox.co.il" target="_blank">leonid@mellanox.co.il</a>]
<br>
<b>Sent:</b> Sunday, January 18, 2009 5:57 AM<br>
<b>To:</b> James Yang; Deepak Gupta; Fab Tillier</span></p>
<div>
<div>
<p><span style="font-size: 10pt;"><br>
<b>Cc:</b> <a href="mailto:ofw@lists.openfabrics.org" target="_blank">ofw@lists.openfabrics.org</a><br>
<b>Subject:</b> RE: [ofw] Expose a vendor defined device in ibbus?</span></p>
</div>
</div>
</div>
<div>
<div>
<p> </p>
<div>
<p><span style="font-size: 10pt; color: blue;">After
first look: why did you put the list of the created devices into Globals (and
not, say, FDO) ?</span></p>
</div>
<div>
<p><span style="font-size: 10pt; color: blue;">How
it will work for multi-home machine ? (several HCA cards)</span></p>
</div>
<div>
<p><span style="font-size: 10pt; color: blue;">Would
anybode like to check it for various partition keys ?</span></p>
</div>
<blockquote style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color blue; border-width: medium medium medium 1.5pt; margin: 5pt 0in 5pt 3.75pt; padding: 0in 0in 0in 4pt;">
<p> </p>
<div style="text-align: center;" align="center">
<hr align="center" size="2" width="100%">
</div>
<p style="margin-bottom: 12pt;"><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;"> <a href="mailto:ofw-bounces@lists.openfabrics.org" target="_blank">ofw-bounces@lists.openfabrics.org</a>
[mailto:<a href="mailto:ofw-bounces@lists.openfabrics.org" target="_blank">ofw-bounces@lists.openfabrics.org</a>]
<b>On Behalf Of </b>James Yang<br>
<b>Sent:</b> Tuesday, January 06, 2009 10:11 PM<br>
<b>To:</b> Deepak Gupta; Fab Tillier<br>
<b>Cc:</b> <a href="mailto:ofw@lists.openfabrics.org" target="_blank">ofw@lists.openfabrics.org</a><br>
<b>Subject:</b> RE: [ofw] Expose a vendor defined device in ibbus?</span></p>
<p><span style="font-size: 10pt; color: navy;">Hi,</span></p>
<p><span style="font-size: 10pt; color: navy;"> </span></p>
<p><span style="font-size: 10pt; color: navy;">Please
review the patch to create user defined devices by reading from registry. By
default there is only one IpoIB device enabled in mlx4_hca.inx file. This patch
will only work for ConnectX.</span></p>
<p><span style="font-size: 10pt; color: navy;"> </span></p>
<p><span style="font-size: 10pt; color: navy;">The
paritition key if set to default to FFFF, I didn't test on other value. And the
Ioctl part to add partition key may also need to be verified.</span></p>
<p><span style="font-size: 10pt; color: navy;"> </span></p>
<p><span style="font-size: 10pt; color: navy;">Thanks,</span></p>
<p><span style="font-size: 10pt; color: navy;">James</span></p>
<p><span style="font-size: 10pt; color: navy;"> </span></p>
<p><span style="font-size: 10pt; color: navy;"> </span></p>
<div>
<div style="text-align: center;" align="center">
<hr align="center" size="2" width="100%">
</div>
<p><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;"> <a href="mailto:mailmeatdkg@gmail.com" target="_blank">mailmeatdkg@gmail.com</a>
[mailto:<a href="mailto:mailmeatdkg@gmail.com" target="_blank">mailmeatdkg@gmail.com</a>]
<b>On Behalf Of </b>Deepak Gupta<br>
<b>Sent:</b> Monday, January 05, 2009 10:51 PM<br>
<b>To:</b> Fab Tillier<br>
<b>Cc:</b> Tzachi Dar; James Yang; <a href="mailto:ofw@lists.openfabrics.org" target="_blank">ofw@lists.openfabrics.org</a><br>
<b>Subject:</b> Re: [ofw] Expose a vendor defined device in ibbus?</span></p>
</div>
<p> </p>
<p style="margin-bottom: 12pt;">Have a gr8 New Year to all the members!!!.<br>
<br>
Do you we any updates on "vendor defined device in ibbus"?<br>
<br>
I wanted to create multiple vnic interfaces irrespective of number of reachable
IOCs.<br>
<br>
Currently I am creating vnic child devices on "root" bus.<br>
Inside VNIC driver, I am looking for arrival GUID_IB_AL_INTERFACE and hence
contacting the IBAL.<br>
But since vnic devices are root enumerated, VNIC driver gets loaded very
earlier in boot phase (Before "Extended Base" group to which IB Stack
drivers belong) and hence VNIC device interfaces are not getting initialized
properly.<br>
<br>
If we are providing a vendor defined device functionality in ibbus in near
future, then it would be worth for me to wait for it.<br>
<br>
Can any one please comment on this.<br>
<br>
Regards<br>
Deepak</p>
<div>
<p>On Mon, Dec 22, 2008 at 1:12 PM, Deepak Gupta <<a href="mailto:deepak.gupta@qlogic.com" target="_blank">deepak.gupta@qlogic.com</a>>
wrote:</p>
<p>All,<br>
<br>
I came across one more question in my mind which are I think is not clear to me
after reading the whole thread.<br>
<br>
In new design being discussed:- <br>
Are we making sure that we can have more than one child devices configured for
the same IOC.<br>
Currently, there is one child device created per IOC discovered.<br>
<br>
Having more than one child device configured for same IOC is a requirement if a
user wants two different ULP interfaces to be created on host side.<br>
Consider a case in which a host is connected to a single IOC and IOC is
connected to a ethernet network via switch.<br>
If there are two different IP subnets then there is a requirement of two
different Ethernet interfaces on the host side too.<br>
<br>
Please let me know if you need more clarification of my question.<br>
<br>
Regards<br>
<span style="color: rgb(136, 136, 136);">Deepak</span></p>
<div>
<div>
<p style="margin-bottom: 12pt;"> </p>
<div>
<p>On Sat, Dec 20, 2008 at 2:19 PM, Deepak Gupta <<a href="mailto:deepak.gupta@qlogic.com" target="_blank">deepak.gupta@qlogic.com</a>>
wrote:</p>
<p style="margin-bottom: 12pt;">Please see below.<br>
<br>
Regards<br>
Deepak</p>
<div>
<div>
<div>
<p>On Sat, Dec 20, 2008 at 12:42 AM, Fab Tillier <<a href="mailto:ftillier@windows.microsoft.com" target="_blank">ftillier@windows.microsoft.com</a>>
wrote:</p>
<div>
<p>> On Wed, Dec 17, 2008 at 11:43 PM, Fab Tillier<br>
> <<a href="mailto:ftillier@windows.microsoft.com" target="_blank">ftillier@windows.microsoft.com</a>>
wrote:</p>
</div>
<div>
<p style="margin-bottom: 12pt;">>> Are there other properties that I have
missed that are needed?<br>
><br>
> We need a way in which devices created should be configured for<br>
> failovers ( ULPs like VNIC, SRP need more configurable failovers).<br>
> Looking at IBAL's code it create the devices based on the reachable<br>
> IOC's and thus failover's are possible across the HCA/ports and not<br>
> across two different IOCs.<br>
> Users can have a case in which two different IOCs connected to same<br>
> physical network/storage (redundancy is provided for high availability)<br>
> and want a failover across the IOCs.</p>
</div>
<p>This would be done via LBFO for network devices, and MPIO for storage
devices. I think having the bus driver report a single IOC that really
maps to two IOCs on the fabric is asking for management problems.
Further, LBFO/MPIO can provide failover between different device types,
so the failover devices don't have to be identical HW.</p>
</div>
</div>
<div>
<p><br>
I don't know about how MPIO works. But for LBFO, BundleID param will have
to be included in extended params then so that user gets the freedom of
bundling different failover configurations.<br>
</p>
</div>
<blockquote style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color rgb(204, 204, 204); border-width: medium medium medium 1pt; margin: 5pt 0in 5pt 4.8pt; padding: 0in 0in 0in 6pt;">
<p><br>
<span style="color: rgb(136, 136, 136);"><br>
-Fab</span></p>
</blockquote>
</div>
<p> </p>
</div>
<p> </p>
</div>
</div>
</div>
<p> </p>
</blockquote>
</div>
</div>
</div>
</div>
<p> </p>
</div>
</div>
<div>
<p>_______________________________________________<br>
ofw mailing list<br>
<a href="mailto:ofw@lists.openfabrics.org" target="_blank">ofw@lists.openfabrics.org</a><br>
<a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw" target="_blank">http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw</a></p>
</div>
</blockquote>
</div>
<p> </p>
</div>
<p> </p>
</div></div></div>
</div>
</div>
<br>_______________________________________________<br>
ofw mailing list<br>
<a href="mailto:ofw@lists.openfabrics.org" target="_blank">ofw@lists.openfabrics.org</a><br>
<a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw" target="_blank">http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw</a><br></blockquote></div></div></div><br>
</div><br>