[openib-general] Re: TSO and IPoIB performance degradation

[Troy Benjegerdes]

On Thu, Apr 27, 2006 at 05:16:29PM -0700, Greg Lindahl wrote:
> On Thu, Apr 27, 2006 at 04:22:40PM -0700, Grant Grundler wrote:
> 
> > Anything preventnig such a gateway from routing SDP to ethernet?
> > Those gateways obviously will grok IB protocols.
> > I'm asking becuase I don't understand/know if there is a real
> > barrier to an IB -> ethernet gateway _without_ IPoIB.
> 
> I don't know if a SDP to ethernet gateway even exists, but I do know
> that it's a lot more work than just an IPoIB to ethernet gateway --
> the gateway is going to have to pass all its data through a TCP stack.
> So I would expect SDP to ethernet to not run very fast, especially on
> a gateway with lots of streams going.

And this is exactly the reason that we should not be playing games with
"infiniband specific" TCP optimzations. If you stay on the IB network,
use SDP or verbs. If you are going to cross networks, you want to be
running the full host TCP stack that has been well tested and is robust
to all the kinds of failures you see crossing networks. This does not
mean that it won't be fast, but you *will* have more overhead than on a
single network fabric.

If someone has a configureation where full TCP processing on
the host is a bottleneck and not the IPoIB to ethernet gateway, then
let's have this discussion again. But I don't believe such a
configuration actually exists anywhere. If you think you have some
problem like this, I would love to be able to run some benchmarks on the
system.