[openib-general] [PATCH for-2.6.18] IB/cma: option to limitMTU to 1K

[Fabian Tillier]

Hi Hal,

On 13 Sep 2006 12:35:58 -0400, Hal Rosenstock <halr at voltaire.com> wrote:
> Hi Fab,
>
> On Wed, 2006-09-13 at 12:39, Fabian Tillier wrote:
> > On 9/13/06, Michael S. Tsirkin <mst at mellanox.co.il> wrote:
> > > Quoting r. Hal Rosenstock <halr at voltaire.com>:
> > > > Subject: Re: [openib-general] [PATCH for-2.6.18] IB/cma: option to limitMTU to 1K
> > > >
> > > > On Wed, 2006-09-13 at 11:57, Michael S. Tsirkin wrote:
> > > > > Tavor systems get better performance with 1K MTU. Since there does
> > > > > not seem to be any way to find out whether the remote system uses Tavor,
> > > > > add an option to limit the MTU globally.
> > > >
> > > > Can't Tavor be determined locally ?
> > >
> > > It can, but we need this for remote tavor as well, anyway.
> > >
> > > > And couldn't the remote end negotiate the MTU down (if Tavor) as well ?
> > >
> > > The way to do this is would be for SA to select 1K MTU if it detects Tavor on one side
> > > and if this does not conflict with MTU selector.
> >
> > You can't do this because the SA doesn't have a way to tell if a path
> > query is going to be used for RC or UD, and IPoIB needs paths with 2K
> > MTU.
>
> Are you referring to IPoIB-CM ?
>
> The patch appears to be for the SA PR request prior to the CM REQ. I
> don't think it affects IPoIB SA PR requests.

I interpreted Michael's comment as suggesting the SA return paths with
a 1K MTU when it detects that either endpoint is Tavor.  The SA has
access to this information based on the vendor ID/device ID in the
node record.

If I understood Michael's comment properly, this will have the side
effect that IPoIB won't work since IPoIB requires 2K MTUs.  As far as
I know, there is no way to specify whether a path is needed for UD vs.
RC in the path query.

I like your suggestion to reject with a smaller MTU.  Seems like the
proper way to handle this, as well as allowing for the retry logic to
be put in the CMA itself so clients don't have to deal with it.

- Fab