[openib-general] Plans for libibverbs 1.0, 1.1 and beyond
ogerlitz at voltaire.com
Thu Feb 23 03:31:56 PST 2006
Gil Bloch wrote:
> I believe we should add support for a resize WQ command (as a part of
> modify QP) to enable changing the WQ size.
> On a very large scale cluster, with many operating QPs, the work queue
> memory consumption might be expansive. Thus the MPI implementation
> should tradeoff for pipelining requests vs. WQ memory consumption. The
> resize WQ will allow on-demand adaptive WQ setting instead of static
> allocation of the memory resource, which I believe can increase
> performance and save memory at the same time.
As others pointed, i think such changes should be driver by the
a) ---real--- need of an app
b) support of as much HW vendors as possible
Did any IB MPI group approached this list expressing a need for this
Moreover, here are some more points that might suggest this feature is
- why does the size of the cluster relates to how many credits rank A in
an mpi job would have on the connection (QP) with rank B?
- who said that (say) eight credits per connection are not excellent for
an IB MPI?
- some MPIs (eg open MPI open connections by demand so only if rank A
attempts to send something to rank B then A connects to B
- what you really might want to resize is an SRQ in case the
implementations want to open connections by demand and add more and more
QPs to the SRQ, and the work on modify SRQ verb was initiated by Mellanox?
- same for resize CQ - the verb exist and also the need for it.
More information about the general