[Openib-windows] [IBAL][Patch] reference counter for IOC obj in IOC PnP manager

[Fabian Tillier]

Hi Yossi,

I looked a bit more at the patch.  First, a minor nit - it's easier to
review patches if they only deal with one problem at a time...

On 11/22/06, Yossi Leybovich <sleybo at mellanox.co.il> wrote:
>
>
> > -----Original Message-----
> > From: openib-windows-bounces at openib.org
> > [mailto:openib-windows-bounces at openib.org] On Behalf Of Fabian Tillier
> > Sent: Wednesday, November 22, 2006 9:20 AM
> > To: Yossi Leybovich
> > Cc: openib-windows at openib.org
> > Subject: Re: [Openib-windows] [IBAL][Patch] reference counter
> > for IOC obj in IOC PnP manager
> >
> > Hi Yossi,
> >
> > On 11/17/06, Yossi Leybovich <sleybo at mellanox.co.il> wrote:
> > >
> > > 2. When connecting SRP target to 2 HCA on the same host.
> > > The PnP events use the IOC GUID as key to insert/remove from the
> > > context_map.
> > > In case we use the same SRP target with the 2 HCAs they
> > both have same
> > > key.
> > >
> > > I change the context_map to use 128 bit key ( and not 64bit
> > key) The
> > > key is construct from IOC GUID and CA GUID This change
> > effect also the
> > > port events and the CA events.
> >
> > Using the CA GUID to store the distinct IOU/IOC objects in
> > the IOC PnP manager's tables is fine.  However, using
> > different unique IDs for the devices when reporting them to
> > the system is not - it is a single device.  The location
> > information is different, but it is the same device.  Thus, I
> > think the unique ID for the IOC should still be the IOC GUID
> > alone, and not concatenated with the CA GUID.
> >
>
> Windows refuse to create 2 SRP devices with the same unique ID , at
> least that what I get when trying to do so.

Ok, I guess that makes some more sense.  I'm a bit rusty (already!)...

> You can see that the IOU device use 128 bits of unique ID (I just copy
> the code from the IOU unique ID) although it's the same IOU node in the
> both HCAs.
> We can change that (and we discuss that in the past ) but right now we
> create IOU device as the HCA son and the IOC is the son of the IOU .

You can't change that, actually.  The port driver will do DMA mappings
to the underlying device.  You cannot just switch HCAs and expect the
DMA mappings to work.  The drivers were initially setup to have the
IOU reported as a child of the "InfiniBand Fabric" device (ibbus.sys),
but DMA mapping required them to become the child of the HCA.

> So the ibiou (of each HCA ) try to create SRP with the same unique ID.
> If we will create the IOC in the systems level (without IOU device ) we
> can create only one IOC for all HCA that connect to the same target.
> This will also require some changes in the SRP driver to add it ability
> to switch from HCA to HCA and it will save the needs for MPIO DDK driver

Like I said, you can't do this if you want to do proper DMA mappings.
MPIO is the way to go.

- Fab