[openib-general] [PATCH for-2.6.18] IB/cma: option to limitMTU to 1K

[Rimmer, Todd]

> From: Michael S. Tsirkin [mailto:mst at mellanox.co.il]
> Sent: Wednesday, September 13, 2006 6:01 PM
> To: Rimmer, Todd
> Cc: Sean Hefty; openib-general at openib.org
> Subject: Re: [openib-general] [PATCH for-2.6.18] IB/cma: option to
> limitMTU to 1K
> 
> Quoting r. Rimmer, Todd <trimmer at silverstorm.com>:

> >
> > Pushing these types of ULP and source/destination specific issues
into
> > the core stack or SM will get very complex very quick.
> 
> It's actually relatively simple.

So here is how it gets complex.  The best MTU needs to be selected for
various combos such as:
Tavor w/IPoIB
Tavor to Storage target with SRP
Tavor to eHCA with SDP
Tavor to PathScale with MPI
Tavor to DDR Arbel with SRP
etc etc

The answer for many of the above combos may not be 1K MTU runs best.
Hence if we try to support this in the SA, it needs to know about all
these subtle combinations.

The IB spec avoids such complex combos by having each Node reports its
MTU capabilities (as well as others like outstanding RDMA reads, etc).

> 
> > Given the issue
> > on the table (Tavor performance) is specific to an older HCA model,
it
> > may not even be that critical since the highest performance
customers
> > have long since moved toward PCIe and DDR fabrics, neither of which
are
> > supported by Tavor.
> 
> All the more reason to pt the simple logic in one place
> and not expect all apprlications to optimize for this hardware.

All the reason to invest in more important requirements, such as SDP
Z-Copy.  Especially since most of the performance critical applications
(Open MPI, Scali MPI, MVAPICH MPI, etc) have already implemented this
optimization.

Todd Rimmer