[ofw][patch][ND provider] Improving latency of ms-mpi

[Fab Tillier]

Hi Leo,

The default setting of 160 bytes that we tested does make a notable improvement on latencies.  While the ND provider implementation can be changed to treat the parameter as in/out, it doesn't change the interface definition, or the client code that may pass an uninitialized variable expecting it to be used only for output.

Thus, changing the provider to make the parameter in/out actually could break client applications because you're changing the contract defined by the interface.

However, the change to automatically inline data does appear to be beneficial (even for bandwidth, strangely enough.)

-Fab

> -----Original Message-----
> From: Leonid Keller [mailto:leonid at mellanox.co.il]
> Sent: Tuesday, August 11, 2009 3:44 PM
> To: Sean Hefty; Fab Tillier; Tzachi Dar
> Cc: ofw at lists.openfabrics.org
> Subject: RE: [ofw][patch][ND provider] Improving latency of ms-mpi
>
> This change adds a new facility.
> Tzachi and me have brought the reasons.
> So it's good from engineer point view.
> I don't know whether we may do it from legal one.
> But we used to change Ibal API and ND sits in the same Open Source Tree
> with IBAL.
> Who is he maintainer of ND provider code ?
> I thought it's Fab.
> Fab, do you approve this API change ?
>
>
> > -----Original Message-----
> > From: Sean Hefty [mailto:sean.hefty at intel.com]
> > Sent: Monday, August 10, 2009 9:05 PM
> > To: Leonid Keller; Fab Tillier; Tzachi Dar
> > Cc: ofw at lists.openfabrics.org
> > Subject: RE: [ofw][patch][ND provider] Improving latency of ms-mpi
> >
> > >This latter parameter should be IN OUT, because the driver takes its
> > >value as a hint.
> > >It really re-calculates it, trying to maximize in the limits of WQE
> > >size.
> >
> > Isn't this a MS defined API?  I don't believe we can change
> > the NDEndpoint APIs at all.
> >
> > - Sean
> >
> >