[Openib-windows] RE: tools support

[Fab Tillier]

> From: Leonid Keller [mailto:leonid at mellanox.co.il]
> Sent: Tuesday, September 13, 2005 2:58 AM
> 
> Hi Fab,
>
> We are going to add support for a number of utilities, managing HCA cards.
> In the first place it's 'flint', which downloads FW into the card.

There's already a tool to burn FW in tools\fw_update, derived from flint.
 
> There are 2 ways of doing that: via gateway in PCI config space of the HCA
> cards, which is supported by the driver, but is slow,

This is what the current tool does, and it is slow (more so on PCI-Express
systems).

> and by direct writing into  CR
> space of the card after mapping it to the address space of the process.

Why does the CR space have to be mapped into user-mode?  Why can't the kernel
driver just write directly into the CR space, and have the user-mode tool pass a
buffer with the firmware image?

> The latter way is not suppported now and i'm going to implement it by
> adding a new vendor call of subtype MAP_CRSPACE (0x0a), which will perform
> the mapping and return a virtual address to the application.
> What do you think about it ?

I'm weary of exposing configuration space to applications without restricting
their use.  I would rather see use of the direct CR space access by the kernel
driver - this allows the driver to enforce policy on what can and can't be
written to the CR space.

> Another requirement - support of "live fishes" (flash burner devices).
> Any our device can be jumpered to work without CR-space in case when it
> has an invalid (or doesn't have at all) FW on it. In that case it has
> device_id greater by one than its original one (e.g. 23109 - for Tavor).
> The burning is performed by configuration cycles.
> How would you suggest to add support for those devices ?

When an HCA registers itself, IBAL creates a CA object to represent that device.
For the driver to support the "live fish" mode, it would simply report itself as
an HCA with no support for any resources except protection domains.  The
protection domain creation simply returns success (but is a noop internally),
and all other resource creation verbs return failure.  This will prevent the
SMI/GSI from loading, but will allow the HCA to be available for use, which
means user-mode processes can see it.  Once user-mode processes can see the
device, the HCA FW update tool should be able to work without issue.

I think it would be great if the FW update interface was simplified - that is,
rather than having user-mode issue series of 4-byte reads or writes, have the
tool supply a large buffer large enough to contain (read) or containing (write)
the full FW image.  The kernel driver would then perform the correct action -
whether through CR space mapped into the kernel or configuration cycles on the
PCI bus.  This removes the need for the FW update tool to know what kind of
device accesses are possible, and relies on the driver to use the optimal
method.

Hopefully this makes sense.  Let me know if I've missed some subtleties about
the FW update process.

Thanks,

- Fab