[Openib-windows] Getting the HCA pdo

[Fabian Tillier]

Hi Jan,

On 9/22/06, Jan Bottorff <jbottorff at xsigo.com> wrote:
>
> I have a kernel driver that uses the IB stack but is not instantiated by the
> IB bus driver, so in not in the PnP tree of the hca.
>
> One of the little details that took some head scratching was how to
> correctly get the physical address of buffers while correctly doing Windows
> DMA. Currently, I have a piece of code that looks something like:
>
>             al_obj_t * internalHCAHandle;
>
>             internalHCAHandle = (al_obj_t *)hcaHandle;
>
>             nextLowerDevice =
> IoGetAttachedDeviceReference((PDEVICE_OBJECT)(internalHCAHandle->p_ci_ca->verbs.p_hca_dev));

Note that IoGetAttachedDeviceReference gets you a reference to the
highest device object in the device stack, not the next lower device.

> This basically does an ugly cast of a hca handle to its internal data
> structure to retrieve the hca pdo device object. This all works fine except
> for the little details that the shape of al_obj_t changes between a checked
> build and free build. I'd love to remove this ugly code and use something
> more "supported".

I agree a cleaner solution is needed.  Other drivers like SDP don't
put themselves in the device tree.

> Can someone suggest a better way to obtain the actual hca device pdo (or
> appropriate lower device object) to pass to IoGetDMAAdapter. It offhand
> seems like the IBAL interface is missing important functionality, as you
> need physical addresses in QP work requests but don't offhand see any calls
> to get a dma adapter object given an hca/port?

There's an existing call to the HCA, ib_ci_call, that lets you do
pretty much anything you want.  This call was created to allow users
to make calls for things outside of the IB spec.  Currently, it is
used to get access to the PCI configuration space and to map it into
user-mode for firmware updates.  It could easilly be expanded to
support a request for the HCA's device object.  That's the solution
with the least impact on the existing code at the moment, but pushes
the implementation into every HCA driver thus potentially duplicating
functionality.

An other alternative is to add a function to the access layer
interface structure used by kernel clients.  I think I prefer this
solution.

There are a number of little issues like this that I'd like to address
better in the long term, so please keep track of what things you
like/dislike about the APIs and usage model, and we'll have a
brainstorming session to find a way to address these.  Some of the
issues are non-trivial to fix, and a lot of it comes from the code not
having been designed specifically for Windows.

- Fab