[ofw] OFA WWG (Windows Working Group) meeting minutes for 8-23-11

[Smith, Stan]

Attendees:

  Leonid Killer (Mellanox)

  Sean Hefty (Intel)

  Eric Lantz (Microsoft); thank you Eric for sending in meeting notes!

  Wu Wenhao (Microsoft)



Network Direct V2:

  Specification published

  V2 provider SW will not be ready for winOFED 3.0 release (perhaps a future point release).



Mellanox ConnectX-3 FDR support:
o   Ready in late August
o   Relatively few source changes



  Leonid believes a code commit into OpenIB SVN tree by August 31st is possible due to minimal changes to mlx4 driver.

  Leonid agreed to producing an SVN branch containing FDR code which interested parties can review.

  Given the small number of changes, perhaps submitting patches to the ofw list for review would be faster for all?



ROCE support:
o   Changes in many components of the stack
·        What is nature of the changes?

Sending individual patches for review will be too time consuming due to extent of changes.
mlx4_bus\* is an entire replacement

    ibat extended to support ROCE; IPoIB changes?

    Ibal mods
·        No way to accept only part of the changes though
§  Lots of changes in the bus driver
o   How to commit changes to svn?
·        One big change in end of Aug
·        Or many small changes which is hard to ETA
·        Stan: Perhaps make a branch for the MLX4 folder structure and content?  Leonid agrees.
§  Then do inspection and bug hunting on that branch



Leonid agreed to create an SVN branch with ROCE specific changes;  with so many changes creating patch files is very time consuming.

Stan requested the ROCE branch be based on the latest OpenIB SVN trunk so ROCE changes in the branch are easily identifiable.
Interested WWG parties will review branch ASAP.



Leonid noted that with so many changes for ROCE it will be difficult to review and commit into the OpenIB SVN tree by the proposed code freeze date of Aug 31st.



Stan suggested the proposed winOFED 3.0 code freeze date be pushed out to September 15th.

Leonid thought that ROCE commits might be able to be finished by 9/15 although he needed to talk with the rest of the Mellanox Windows team.



Mellanox (binary only) Ethernet driver required for ROCE support:


o   Including binary driver is contrary to philosophy of OSS
o   In the past, ND provider was taken as a binary as a temporary measure only with commitment from MS to check in the ND source when the MS-OFA legal process on contributions was completed.
o   Stan:  Suggest Mellanox make the Ethernet binary available at their website but not in winOFED.
·        User would have to download from 2 locations to use WinOFED ROCE
·        But that driver requires a totally different MELL bus driver when using Ethernet
§  The driver depends on Mellanox bus driver AND the IB stack (including IBAL) for ROCE
o   Leonid would like to discuss in email thread with Uri & others to which Stan agrees



Due to Intellectual Property disclosed in the Mellanox ROCE Ethernet driver source for Windows, Mellanox proposed they supply a binary only driver to be included in the winOFED 3.0 release.

After some offline discussions, all members of the WWG except Mellanox agreed including a binary only driver in the winOFED 3.0 release was an unacceptable solution as binary only inclusion violated the intent of the OFA open-source project. The issue being open-source binaries are re-creatable from source located in the open-source project source repository.

Leonid correctly pointed out that a binary only inclusion had been done before with the Microsoft ND provider.

Eric Lantz pointed out the Microsoft ND provider binary was included only because MS legal and OFA legal were still in process of resolving legal wording and Microsoft agreed upfront to SVN commit the ND provider sources as soon as MS legal would let them;  the ND provider sources were svn committed shortly after the legal issues were resolved.

Stan further pointed out how disruptive binary inclusion was to the winOFED build process (numerous temporary modifications to the build process including manual hands-on steps).



Mellanox had previously stated the Mellanox Ethernet driver sources would not be svn committed due to exposure of IP in the source.

Stan noted the same IP issues were present in the Linux OFED ROCE Ethernet driver sources, so why is the Windows driver different?

None the less, the WWG agreed a binary only ROCE Ethernet driver solution was unacceptable.

Leonid asked if winOFED wanted ROCE support, Stan replied yes although the WWG is willing to live without ROCE support if there is no alternative beyond a binary only ROCE Ethernet driver solution.

Stan further suggested Mellanox could publish their Windows ROCE Ethernet driver on the Mellanox website; similar to how Mellanox HCA firmware tools are published.

When winOFED customers wish to use ROCE hardware over Ethernet, they would install winOFED and then download/install the Mellanox Ethernet driver; (a win-win story).

Stan suggested it's in Mellanox' s best interest to be able to update the Mellanox Ethernet driver binaries and WHQL said Ethernet driver independent of winOFED involvement.

Leonid then pointed out the ROCE mlx4_bus driver has link time dependencies on the Mellanox Ethernet driver.

Everyone, outside of Mellanox, agreed a link time dependency on the Mellanox Ethernet driver was unacceptable in that the winOFED stack would not load without the Mellanox binary only Ethernet driver being present.



Sean requested more information about other link time dependencies the ROCE code base had on other winOFED modules.



Stan proposed a modification to the ROCE mlx4_bus driver such that when the Mellanox Ethernet driver is loaded it's Interface is exported to the ROCE mlx4_bus driver thus removing the winOFED stack link time dependency on the Mellanox ROCE Ethernet driver.

Leonid agreed to start an ofw email thread discussing how the link time dependency on the Mellanox Ethernet driver could be removed and how the mlx4_bus driver would then receive notification when the Mellanox ROCE Ethernet driver interface was available. The interface notification mechanism would be similar to how IBAL and Winverbs receive notification from HCA drivers today.



winOFED 3.0 release schedule:
o   Stan proposed moving the WinOFED 3.0 code free to Sept15
·        To get ROCE and FDR into v3.0
o   Ready to start the RC1 test pass on Sept19
·        Desirable to have hardware in place for ROCE and FDR testing in the MS labs by this date
§  How do MS, Intel, ?others? Get access to FDR hardware in Sept?
§  ROCE does run on ConnectX-2 hardware
·        Also verify the download of Mellanox Ethernet driver bits from their website works with these WinOFED v3.0 bits as part of this testing



Opens:



·        Eric Lantz asked the question of how FDR hardware testing was to be accomplished in that MS and Intel did not have access to FDR capable hardware.
Perhaps Mellanox can allocate time on their for-customer-use clusters which have FDR hardware installed?



·        Eric further noted MS success on using cable plug converters going from QSFP (40gb) IB connectors to 10gb Ethernet switch (SFP?) plugs.



·        Eric also questioned as to how ROCE was to be tested. Stan pointed out he had Connect-X2 hardware which could be cabled back-2-back to test ROCE functionality.










-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofw/attachments/20110823/b84345d2/attachment.html>