[openib-general] Plans for libibverbs 1.0, 1.1 and beyond

[Gil Bloch]

Roland,

I believe we should add support for a resize WQ command (as a part of
modify QP) to enable changing the WQ size.
On a very large scale cluster, with many operating QPs, the work queue
memory consumption might be expansive. Thus the MPI implementation
should tradeoff for pipelining requests vs. WQ memory consumption. The
resize WQ will allow on-demand adaptive WQ setting instead of static
allocation of the memory resource, which I believe can increase
performance and save memory at the same time. 

Regards,
Gil Bloch


> -----Original Message-----
> From: openib-general-bounces at openib.org [mailto:openib-general-
> bounces at openib.org] On Behalf Of Roland Dreier
> Sent: Thursday, February 16, 2006 9:18 PM
> To: openib-general at openib.org
> Subject: [openib-general] Plans for libibverbs 1.0, 1.1 and beyond
> 
> I thought it might be helpful to give an informal roadmap of my plans
> for libibverbs.  As I said, I hope to have a libibverbs 1.0 release
> out in three weeks or so.  Once that happens, I plan to do 1.0.x
> maintenance releases "as needed."
> 
> At the same time, I have some ideas for libibverbs 1.1.  I think that
> I can get my ideas done in six months or so, which seems like a good
> interval between major releases.  My ideas (also listed in the
> libibverbs README) are the following; I'd like to hear what other
> people's plans for libibverbs work are so that we can figure out if
> those projects fit into a 1.1 release, and when we expect to land the
> new features.
> 
>  * Implement memory window (MW) support.  This will break the
>    device driver ABI, because new methods will need to be added to
>    struct ibv_context_ops.
> 
>  * Implement the reregister memory region (MR) verb.  We will add an
>    extension to the IB spec to allow the application to indicate that
>    the region is only being extended, and that operations in progress
>    should _not_ fail (contrary to the IB spec, which states that
>    reregister must be implemented so that it behaves equivalently to a
>    deregister followed by a register).  This will break the device
>    driver ABI, because a new method will need to be added to struct
>    ibv_context_ops.
> 
>  * Eliminate the dependency on libsysfs by implementing the required
>    sysfs handling directly.  This will break the API, because the dev
>    and ibdev members of struct ibv_device will be removed.  It will
>    also break the device driver ABI, because the signature of the
>    driver initialization function will change.  The driver
>    initialization function will be changed as part of this work; this
>    has the added benefit of allowing us to choose a better name than
>    "openib_driver_init."
> 
> I'm also thinking of moving my libibverbs and libmthca development
> trees to git (most likely hosted at kernel.org).  This has the
> drawback of moving their development repositories out of the common
> openib.org svn tree.  However, it will make handling 1.0, 1.1 and
> feature development branches much easier.  I'd like to hear opinions
> on this before I make a decision.
> 
>  - R.
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit
http://openib.org/mailman/listinfo/openib-
> general