[Openib-windows] [RFC] IRP-based verbs

[Fab Tillier]

> From: Tzachi Dar [mailto:tzachid at mellanox.co.il]
> Sent: Monday, September 12, 2005 2:24 AM
> 
> Hi Fab,
>
> Before we get down to the details of your proposal we would like to clarify
> our general opinion in this area.
>
> 1) Enabling asynchronous calls to the command interface is something that we
> believe that should be done in the mid-term schedule.

If we don't tackle this soon it will become too hard to do and will never be
done.  I would like to see the design of this agreed upon before PathScale
starts their development so that they don't have to implement to a flawed driver
model, just to change models a few months later.  This is one of the factors
driving this change sooner rather than later.  The other factors have to do with
simplifying the code base which should yield better stability and
maintainability.

> 2) We don't see any way to achieve this goal without significant changes to
> the low level driver (currently under rewrite). A preliminary version of the
> driver should be available on November.

If you're re-writing the lower level driver, why not do it once and do it right?
Why do it using a driver model that we know is flawed and must change?  The idea
of making verbs asynchronous in the kernel is not new, and I have mentioned it
to you ever since we started working together (almost 6 months ago!).
 
> 3) We will be happy to accept patches from you that will turn all verbs (in
> the driver) to by asynchronous. (We will probably be able to start looking at
> the patches on January next year).

This needs to happen sooner than January.  PathScale expressed interest in
developing an HCA driver for OpenIB Windows at the last workshop.  We shouldn't
impose a broken driver model on them.  I think that rather than focusing on new
feature content (like SDP), it is more important to focus on making the existing
code base correct for the operating system we're targeting.
 
> 4) As this is a very big change to the entire stack, it seems that the
> stability of the stack will be harmed. Therefore, we ask that you do the
> change in a different branch.

I was planning on working in a different branch - I didn't expect to do this in
the mainline due to the extent of the changes.  However, once I start it will be
my priority to get this done.  The design for this is going to move forward, and
I hope to get feedback from you and your team.  Once this is implemented and
working, I plan to merge it back into the mainline before tackling changes to
the client API.

> 5) Also, to my understanding, this change will cause changes in IBAL, and all
> ULPs. As a result I believe that we should make a design review of the changes
> so that we make sure to get all the benefits from it.

That's what RFCs are for.  Initially, the design changes are entirely internal
to IBAL, and hidden from clients.  This will allow us to get the HCA driver
interface worked out and functional before tackling the ULPs.
 
> 6) We should probably schedule a phone meeting to make sure that we all
> understand each other.

This needs to be worked in the open so that all parties that have a vested
interest (all subscribed to the openib-windows mailing list) have a chance to
comment.  In fact, it would be nice to see the SDP bits and memfree HCA driver
developed in the open so that they can be commented on.  Dropping completed
items into the tree makes it very difficult to grasp the work that was done, as
well as making it much harder to affect changes to the design.

That said I'm happy to have a phone conference with your team if you see such a
need.  

- Fab