SPAM Re: SPAM Re: [ofw] Expose a vendor defined device in ibbus?

Tue Feb 3 01:47:03 PST 2009

Leonid,

Forgive me if I am not following you correctly.

IIRC, you are talking about "\device\ibal" which is created whenever first
IB device is encountered.

I am not thinking of creating this new device in AddDevice routine of ibiou.
We will create the device (let's say \Device\VNICCONFIG) at DriverEntry and
is not tied to any instance of IOU FDO.

We can maintain a global list of all ioc_mgrs, to which we can reference to
obtain the list of all IOCs on each time we get a specific IOCTL from user
space.

Regards
Deepak

On Mon, Feb 2, 2009 at 8:46 PM, Leonid Keller <leonid at mellanox.co.il> wrote:

>  What will happen if the first created device will be disabled by user
> from Device Manager ?
> AFAIR, only the first device you attach to HCA and maybe, only for the
> first device create a name...
> We have such kind of problem with IBAL today.
> Only the first instance of IBAL has its name. So the disabling of the first
> HCA causes disappearing of IBAL device and all IB applications stop working
> ...
>
>  ------------------------------
> *From:* ofw-bounces at lists.openfabrics.org [mailto:
> ofw-bounces at lists.openfabrics.org] *On Behalf Of *Deepak Gupta
> *Sent:* Friday, January 30, 2009 8:54 AM
> *To:* Smith, Stan
> *Cc:* ofw at lists.openfabrics.org
> *Subject:* ***SPAM*** Re: [ofw] Expose a vendor defined device in ibbus?
>
> Stan,
>
> Thanks for response.
>
> Yes you are correct, we will create a named device object with user space
> visible symbolic link to it.
> We will send IOCTLs to the this device for child device creations (as
> configured by user).
> As explained in previous mails by Alex and I, we will need this feature
> QLogic EVIC to give user a configuration option.
> If there are concerns over user configuration of SRP PDO's, then we will
> restrict the implementation to QLogic EVIC's IOC's only.
> And device creations for other ULP's (SRP) will behave as it is used to
> behave.
>
> Regards
> Deepak
>
>
> On Thu, Jan 29, 2009 at 10:45 PM, Smith, Stan <stan.smith at intel.com>wrote:
>
>>  Please see inline comments.
>>
>>  ------------------------------
>> *From:* ofw-bounces at lists.openfabrics.org [mailto:
>> ofw-bounces at lists.openfabrics.org] *On Behalf Of *Deepak Gupta
>> *Sent:* Tuesday, January 27, 2009 7:19 AM
>> *To:* ofw at lists.openfabrics.org
>> *Subject:* ***SPAM*** Re: [ofw] Expose a vendor defined device in ibbus?
>>
>>
>> Since I didn't receive any concrete response from the list about the
>> patch, I still have following questions unanswered:--
>>
>> 1) I would like to know who is current owner of ibiou and get a feedback
>> on whether above patch is acceptable.
>> Or Do you suggest some modifications on it ?
>>
>> See trunk\docs\maintainers.txt or
>> http://www.openfabrics.org/downloads/WinOF/README
>>
>> 2) If we are ok with this patch then what time frame should we set for it.
>> Should we target it for coming 2.1 release or for post 2.1 releases?
>>
>> WinOF 2.1 freezes functionality in April; see
>> http://www.openfabrics.org/downloads/WinOF/WinOF_Roadmap.txt
>>
>   <http://www.openfabrics.org/downloads/WinOF/WinOF_Roadmap.txt>
>>
>> 3) Also, I want to move ahead for implementing IOCTL layer in ibiou for
>> creation of child devices and IOC listings on user request. Is it acceptable
>> to implement this IOCTL layer for device creations and IOC listings in ibiou
>> and you suggest to achieve the same through some other means?
>>
>> By IOCTL layer are you implying the creation of a user visible device to
>> which a user-app will open->ioctl-> close?
>> Said IOC device is implemented in ibiou.sys driver?
>>
>>
>> *I want to reiterate the purpose of this patch:--*
>>
>> This patch is for creating multiple PDO's for a HCA to IOC path as
>> configured by the user in INF file.
>> It will help in having multiple sessions with the IOC and hence will give
>> some user configurations for fail over.
>>
>> *Behavior change from current ibiou implementation:--*
>>
>> Currently ibiou simply discovers the IOCs and creates child devices for
>> the IOCs discovered.
>> This patch will create multiple child devices for HCA to IOC paths as
>> configured by the user in INF file.
>> If there are no child device configurations for a particular IOC then no
>> devices will be created for that IOC and hence will not appear in device
>> manager's device tree.
>>
>> I'm no IOU expert so please bare with me on this...
>>
>> Your implication here is the assumption of prior knowledge of which IOCs
>> will be present on the fabric.
>> If I have an OFED SRP target then I will need a specific entry in the
>> ibiou.inf file to describe the SRP target?
>> Seems to be rather cumbersome. Particularly from a package installation
>> perspective. If a .msi installer chooses the SRP option, then how does said
>> option get into the ibiou.inf file? Is the SRP entry always present in the
>> .inf file?
>> If the SRP device does not appear then how will it's device driver be
>> installed?
>>
>> How would IPoIB work?
>>
>> All in all, if I am understanding your design, it sounds like you are
>> defeating the fabric discovery functionality which I believe to be a very
>> desirable feature.
>>
>>
>> This is just a first draft patch to show the usability and we have plans
>> to implement IOCTLs in ibiou for child device creations and for listing the
>> reachable IOC PATHs so that a user application can be written which can list
>> the IOCs and local HCAs to the user and user can send IOCTLs to create
>> devices for the ULPs.
>>
>>
>> Regards
>> Deepak
>>
>>
>>
>>
>>   On Tue, Jan 27, 2009 at 7:09 PM, Deepak Gupta <deepak.gupta at qlogic.com>wrote:
>>
>>> Forgot to include the list.
>>>
>>> Regards
>>> Deepak
>>>
>>>  ---------- Forwarded message ----------
>>> From: Deepak Gupta <deepak.gupta at qlogic.com>
>>> Date: Sat, Jan 24, 2009 at 5:07 PM
>>> Subject: Re: [ofw] Expose a vendor defined device in ibbus?
>>>  To: Fab Tillier <ftillier at windows.microsoft.com>
>>>
>>>
>>> Fab, Nice to see your response.
>>> Please see below.
>>>
>>>  On Fri, Jan 23, 2009 at 11:09 PM, Fab Tillier <
>>> ftillier at windows.microsoft.com> wrote:
>>>
>>>>  I don't have time to look at this in depth, but I can tell you from
>>>> past experience that having a single PDO for an IOC leads to trouble if you
>>>> ever have multiple HCAs in the system.  If you have a miniport driver (like
>>>> NDIS or StorPort), the port driver takes care of DMA mappings of
>>>> user-provided buffers.  That DMA mapping goes down to the PCI driver for the
>>>> particular HCA, so you could potentially have a mapping that isn't valid for
>>>> one of the multiple HCAs in the system.  It's best IMO to have a PDO per IOC
>>>> per HCA (this allows automatic path migration to work in multi-port HCAs),
>>>> and push failover between IOCs to a higher level (LBFO/MPIO).
>>>>
>>>
>>> I had the same concern while implementing the patch.
>>> So I tried to dig into the stack till h/w drivers and it came to me that
>>> all DMA mappings are being
>>> handled by PCI driver.
>>> So this patch creates PDO's per IOC per HCA.
>>> User can specify the child device descriptions in INF file (later on we
>>> can think of having a IOCTL interface in ibiou) and HCA to IOC path.
>>> ibiou will create child PDO's only if HCA to IOC path is present.
>>>
>>> Any more suggestions are welcome!!!
>>>
>>> Regards
>>> Deepak
>>>
>>>
>>>>
>>>>
>>>> -Fab
>>>>
>>>>
>>>>
>>>> *From:* mailmeatdkg at gmail.com [mailto:mailmeatdkg at gmail.com] *On Behalf
>>>> Of *Deepak Gupta
>>>> *Sent:* Friday, January 23, 2009 6:41 AM
>>>> *To:* James Yang
>>>> *Cc:* Leonid Keller; Fab Tillier; ofw at lists.openfabrics.org
>>>>
>>>> *Subject:* Re: [ofw] Expose a vendor defined device in ibbus?
>>>>
>>>>
>>>>
>>>> Did any one get the chance to have a look at the patch?
>>>>
>>>> I am really concerned with the tight coupling of representing each
>>>> single IOC as a single PDO.
>>>>
>>>> Regards
>>>> Deepak
>>>>
>>>>
>>>>  On Wed, Jan 21, 2009 at 5:55 PM, Deepak Gupta <deepak.gupta at qlogic.com>
>>>> wrote:
>>>>
>>>> All,
>>>>
>>>> Based on James patch of creating user defined devices tied to local HCA
>>>> ports, I have created a first draft patch attached herewith
>>>> to allow "ibiou" to create user defined devices which will associate
>>>> with IOC paths.
>>>>
>>>> This patch does the following:-
>>>>
>>>> Now ibiou will not create child devices on it's own when it discovers a
>>>> IOC.
>>>> Instead it will create child devices only when it is instructed to do
>>>> so.
>>>> For the sake of this patch, currently it uses name/description of the
>>>> devices hard coded into INF file of ibiou driver.
>>>> Later on we can implement a IOCTL to pass this information to the driver
>>>> (more below)
>>>>
>>>> It will make a listing of devices to be created in it's DriverEntry
>>>> routine.
>>>> When it will get IOC_PNP_ADD pnp events, it will check in device list
>>>> prepare
>>>> earlier. If it finds device(s) in it's list which corresponds to the new
>>>> IOC
>>>> discovered then it will create those child devices and a similar
>>>> mechanism
>>>> will happen when ibiou will get IOC_PNP_ADD event.
>>>>
>>>> This more of a initial patch for giving an insight into what we want to
>>>> achieve.
>>>> On a longer run we are thinking of providing a IOCTL support in ibiou
>>>> driver for creating child devices (like vnic/srp/etc) and
>>>> differnet vendor specific applications can create those devices and have
>>>> their driver loaded on top of that.
>>>>
>>>> Following is the advantage of this functionality:-
>>>>
>>>>    Earlier there was only one device created per IOC on host side.
>>>>    Now there can be multiple devices on host side per IOC as configured
>>>> in the registry.
>>>>    It will be many to one function i.e there can be many devices on host
>>>> side which will target to the same IOC.
>>>>    It will be like having different sessions with the same IOC.
>>>>
>>>> Please have a look at it and let me know of your
>>>> comments/suggestions/feeback.
>>>>
>>>> NOTE--> Device descriptions in the INF file of this patch creates
>>>> devices that corresponds to a IOC PATH (CAGUID and IOCGUID).
>>>>         Those values are local to my machine and you should change it
>>>> according to your fabric.
>>>>
>>>> Regards
>>>> Deepak
>>>>
>>>>
>>>>   On Tue, Jan 20, 2009 at 1:37 AM, James Yang <jyang at xsigo.com> wrote:
>>>>
>>>>   The proposal is to use registry key to define vendor devices, and the
>>>> registry key is global to the driver. The assumption is that multiple HCA
>>>> cards will have the same vendor defined devices. It cannot support one HCA
>>>> with vendor-A device, and the other HCA with vendor-B device, in the same
>>>> system. At default all hcas will have IPoIB as child devices.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> James
>>>>
>>>>
>>>>  ------------------------------
>>>>
>>>> *From:* Leonid Keller [mailto:leonid at mellanox.co.il]
>>>> *Sent:* Sunday, January 18, 2009 5:57 AM
>>>> *To:* James Yang; Deepak Gupta; Fab Tillier
>>>>
>>>>
>>>> *Cc:* ofw at lists.openfabrics.org
>>>> *Subject:* RE: [ofw] Expose a vendor defined device in ibbus?
>>>>
>>>>
>>>>
>>>> After first look: why did you put the list of the created devices into
>>>> Globals (and not, say, FDO) ?
>>>>
>>>> How it will work for multi-home machine ? (several HCA cards)
>>>>
>>>> Would anybode like to check it for various partition keys ?
>>>>
>>>>
>>>>  ------------------------------
>>>>
>>>> *From:* ofw-bounces at lists.openfabrics.org [mailto:
>>>> ofw-bounces at lists.openfabrics.org] *On Behalf Of *James Yang
>>>> *Sent:* Tuesday, January 06, 2009 10:11 PM
>>>> *To:* Deepak Gupta; Fab Tillier
>>>> *Cc:* ofw at lists.openfabrics.org
>>>> *Subject:* RE: [ofw] Expose a vendor defined device in ibbus?
>>>>
>>>> Hi,
>>>>
>>>>
>>>>
>>>> Please review the patch to create user defined devices by reading from
>>>> registry. By default there is only one IpoIB device enabled in mlx4_hca.inx
>>>> file. This patch will only work for ConnectX.
>>>>
>>>>
>>>>
>>>> The paritition key if set to default to FFFF, I didn't test on other
>>>> value. And the Ioctl part to add partition key may also need to be verified.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> James
>>>>
>>>>
>>>>
>>>>
>>>>  ------------------------------
>>>>
>>>> *From:* mailmeatdkg at gmail.com [mailto:mailmeatdkg at gmail.com] *On Behalf
>>>> Of *Deepak Gupta
>>>> *Sent:* Monday, January 05, 2009 10:51 PM
>>>> *To:* Fab Tillier
>>>> *Cc:* Tzachi Dar; James Yang; ofw at lists.openfabrics.org
>>>> *Subject:* Re: [ofw] Expose a vendor defined device in ibbus?
>>>>
>>>>
>>>>
>>>> Have a gr8 New Year to all the members!!!.
>>>>
>>>> Do you we any updates on "vendor defined device in ibbus"?
>>>>
>>>> I wanted to create multiple vnic interfaces irrespective of number of
>>>> reachable IOCs.
>>>>
>>>> Currently I am creating vnic child devices on "root" bus.
>>>> Inside VNIC driver, I am looking for arrival GUID_IB_AL_INTERFACE and
>>>> hence contacting the IBAL.
>>>> But since vnic devices are root enumerated, VNIC driver gets loaded very
>>>> earlier in boot phase (Before "Extended Base" group to which IB Stack
>>>> drivers belong) and hence VNIC device interfaces are not getting initialized
>>>> properly.
>>>>
>>>> If we are providing a vendor defined device functionality in ibbus in
>>>> near future, then it would be worth for me to wait for it.
>>>>
>>>> Can any one please comment on this.
>>>>
>>>> Regards
>>>> Deepak
>>>>
>>>> On Mon, Dec 22, 2008 at 1:12 PM, Deepak Gupta <deepak.gupta at qlogic.com>
>>>> wrote:
>>>>
>>>> All,
>>>>
>>>> I came across one more question in my mind which are I think is not
>>>> clear to me after reading the whole thread.
>>>>
>>>> In new design being discussed:-
>>>> Are we making sure that we can have more than one child devices
>>>> configured for the same IOC.
>>>> Currently, there is one child device created per IOC discovered.
>>>>
>>>> Having more than one child device configured for same IOC is a
>>>> requirement if a user wants two different ULP interfaces to be created on
>>>> host side.
>>>> Consider a case in which a host is connected to a single IOC and IOC is
>>>> connected to a ethernet network via switch.
>>>> If there are two different IP subnets then there is a requirement of two
>>>> different Ethernet interfaces on the host side too.
>>>>
>>>> Please let me know if you need more clarification of my question.
>>>>
>>>> Regards
>>>> Deepak
>>>>
>>>>
>>>>
>>>> On Sat, Dec 20, 2008 at 2:19 PM, Deepak Gupta <deepak.gupta at qlogic.com>
>>>> wrote:
>>>>
>>>> Please see below.
>>>>
>>>> Regards
>>>> Deepak
>>>>
>>>> On Sat, Dec 20, 2008 at 12:42 AM, Fab Tillier <
>>>> ftillier at windows.microsoft.com> wrote:
>>>>
>>>> > On Wed, Dec 17, 2008 at 11:43 PM, Fab Tillier
>>>> > <ftillier at windows.microsoft.com> wrote:
>>>>
>>>> >> Are there other properties that I have missed that are needed?
>>>> >
>>>> > We need a way in which devices created should be configured for
>>>> > failovers ( ULPs like VNIC, SRP need more configurable failovers).
>>>> > Looking at IBAL's code it create the devices based on the reachable
>>>> > IOC's and thus failover's are possible across the HCA/ports and not
>>>> > across two different IOCs.
>>>> > Users can have a case in which two different IOCs connected to same
>>>> > physical network/storage (redundancy is provided for high
>>>> availability)
>>>> > and want a failover across the IOCs.
>>>>
>>>> This would be done via LBFO for network devices, and MPIO for storage
>>>> devices.  I think having the bus driver report a single IOC that really maps
>>>> to two IOCs on the fabric is asking for management problems.  Further,
>>>> LBFO/MPIO can provide failover between different device types, so the
>>>> failover devices don't have to be identical HW.
>>>>
>>>>
>>>> I don't know about how MPIO works. But for LBFO,  BundleID param will
>>>> have to be included in extended params then so that user gets the freedom of
>>>> bundling  different failover configurations.
>>>>
>>>>
>>>>
>>>>
>>>> -Fab
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> ofw mailing list
>>>> ofw at lists.openfabrics.org
>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> ofw mailing list
>>>> ofw at lists.openfabrics.org
>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>>>>
>>>
>>>
>>>
>>
>> _______________________________________________
>> ofw mailing list
>> ofw at lists.openfabrics.org
>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>>
>
>
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofw/attachments/20090203/dca9baa1/attachment.html>

***SPAM*** Re: ***SPAM*** Re: [ofw] Expose a vendor defined device in ibbus?

SPAM Re: SPAM Re: [ofw] Expose a vendor defined device in ibbus?