<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=US-ASCII">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2654.45">
<TITLE>RE: tools support</TITLE>
</HEAD>
<BODY>
<BR>
<BR>
<P><FONT SIZE=2>> -----Original Message-----</FONT>
<BR><FONT SIZE=2>> From: Fab Tillier [<A HREF="mailto:ftillier@silverstorm.com">mailto:ftillier@silverstorm.com</A>]</FONT>
<BR><FONT SIZE=2>> Sent: Tuesday, September 13, 2005 8:03 PM</FONT>
<BR><FONT SIZE=2>> To: 'Leonid Keller'</FONT>
<BR><FONT SIZE=2>> Cc: openib-windows@openib.org</FONT>
<BR><FONT SIZE=2>> Subject: RE: tools support</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> > From: Leonid Keller [<A HREF="mailto:leonid@mellanox.co.il">mailto:leonid@mellanox.co.il</A>]</FONT>
<BR><FONT SIZE=2>> > Sent: Tuesday, September 13, 2005 2:58 AM</FONT>
<BR><FONT SIZE=2>> > </FONT>
<BR><FONT SIZE=2>> > Hi Fab,</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > We are going to add support for a number of utilities, </FONT>
<BR><FONT SIZE=2>> managing HCA cards.</FONT>
<BR><FONT SIZE=2>> > In the first place it's 'flint', which downloads FW into the card.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> There's already a tool to burn FW in tools\fw_update, derived </FONT>
<BR><FONT SIZE=2>> from flint.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> > There are 2 ways of doing that: via gateway in PCI config </FONT>
<BR><FONT SIZE=2>> space of the HCA</FONT>
<BR><FONT SIZE=2>> > cards, which is supported by the driver, but is slow,</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> This is what the current tool does, and it is slow (more so </FONT>
<BR><FONT SIZE=2>> on PCI-Express</FONT>
<BR><FONT SIZE=2>> systems).</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> > and by direct writing into CR</FONT>
<BR><FONT SIZE=2>> > space of the card after mapping it to the address space of </FONT>
<BR><FONT SIZE=2>> the process.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Why does the CR space have to be mapped into user-mode? Why </FONT>
<BR><FONT SIZE=2>> can't the kernel</FONT>
<BR><FONT SIZE=2>> driver just write directly into the CR space, and have the </FONT>
<BR><FONT SIZE=2>> user-mode tool pass a</FONT>
<BR><FONT SIZE=2>> buffer with the firmware image?</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>[LK] Because user tools may have very complicate logic of how to work with device.</FONT>
<BR><FONT SIZE=2>[it also answers on your below question]</FONT>
<BR><FONT SIZE=2>We are not going to expose this interface, but we need to provide effective tools.</FONT>
<BR><FONT SIZE=2>'flint' is only one example of a tool. </FONT>
</P>
<P><FONT SIZE=2>> > The latter way is not suppported now and i'm going to </FONT>
<BR><FONT SIZE=2>> implement it by</FONT>
<BR><FONT SIZE=2>> > adding a new vendor call of subtype MAP_CRSPACE (0x0a), </FONT>
<BR><FONT SIZE=2>> which will perform</FONT>
<BR><FONT SIZE=2>> > the mapping and return a virtual address to the application.</FONT>
<BR><FONT SIZE=2>> > What do you think about it ?</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> I'm weary of exposing configuration space to applications </FONT>
<BR><FONT SIZE=2>> without restricting</FONT>
<BR><FONT SIZE=2>> their use. I would rather see use of the direct CR space </FONT>
<BR><FONT SIZE=2>> access by the kernel</FONT>
<BR><FONT SIZE=2>> driver - this allows the driver to enforce policy on what can </FONT>
<BR><FONT SIZE=2>> and can't be</FONT>
<BR><FONT SIZE=2>> written to the CR space.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> > Another requirement - support of "live fishes" (flash </FONT>
<BR><FONT SIZE=2>> burner devices).</FONT>
<BR><FONT SIZE=2>> > Any our device can be jumpered to work without CR-space in </FONT>
<BR><FONT SIZE=2>> case when it</FONT>
<BR><FONT SIZE=2>> > has an invalid (or doesn't have at all) FW on it. In that </FONT>
<BR><FONT SIZE=2>> case it has</FONT>
<BR><FONT SIZE=2>> > device_id greater by one than its original one (e.g. 23109 </FONT>
<BR><FONT SIZE=2>> - for Tavor).</FONT>
<BR><FONT SIZE=2>> > The burning is performed by configuration cycles.</FONT>
<BR><FONT SIZE=2>> > How would you suggest to add support for those devices ?</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> When an HCA registers itself, IBAL creates a CA object to </FONT>
<BR><FONT SIZE=2>> represent that device.</FONT>
<BR><FONT SIZE=2>> For the driver to support the "live fish" mode, it would </FONT>
<BR><FONT SIZE=2>> simply report itself as</FONT>
<BR><FONT SIZE=2>> an HCA with no support for any resources except protection </FONT>
<BR><FONT SIZE=2>> domains. The</FONT>
<BR><FONT SIZE=2>> protection domain creation simply returns success (but is a </FONT>
<BR><FONT SIZE=2>> noop internally),</FONT>
<BR><FONT SIZE=2>> and all other resource creation verbs return failure. This </FONT>
<BR><FONT SIZE=2>> will prevent the</FONT>
<BR><FONT SIZE=2>> SMI/GSI from loading, but will allow the HCA to be available </FONT>
<BR><FONT SIZE=2>> for use, which</FONT>
<BR><FONT SIZE=2>> means user-mode processes can see it. Once user-mode </FONT>
<BR><FONT SIZE=2>> processes can see the</FONT>
<BR><FONT SIZE=2>> device, the HCA FW update tool should be able to work without issue.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>[LK] Is there an issue with GUID ?</FONT>
<BR><FONT SIZE=2>HCA driver uses GUID as an identifier of the card.</FONT>
<BR><FONT SIZE=2>It can't find out it in this case.</FONT>
</P>
<P><FONT SIZE=2>> I think it would be great if the FW update interface was </FONT>
<BR><FONT SIZE=2>> simplified - that is,</FONT>
<BR><FONT SIZE=2>> rather than having user-mode issue series of 4-byte reads or </FONT>
<BR><FONT SIZE=2>> writes, have the</FONT>
<BR><FONT SIZE=2>> tool supply a large buffer large enough to contain (read) or </FONT>
<BR><FONT SIZE=2>> containing (write)</FONT>
<BR><FONT SIZE=2>> the full FW image. The kernel driver would then perform the </FONT>
<BR><FONT SIZE=2>> correct action -</FONT>
<BR><FONT SIZE=2>> whether through CR space mapped into the kernel or </FONT>
<BR><FONT SIZE=2>> configuration cycles on the</FONT>
<BR><FONT SIZE=2>> PCI bus. This removes the need for the FW update tool to </FONT>
<BR><FONT SIZE=2>> know what kind of</FONT>
<BR><FONT SIZE=2>> device accesses are possible, and relies on the driver to use </FONT>
<BR><FONT SIZE=2>> the optimal</FONT>
<BR><FONT SIZE=2>> method.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Hopefully this makes sense. Let me know if I've missed some </FONT>
<BR><FONT SIZE=2>> subtleties about</FONT>
<BR><FONT SIZE=2>> the FW update process.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Thanks,</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> - Fab</FONT>
<BR><FONT SIZE=2>> </FONT>
</P>
</BODY>
</HTML>