[openib-general] IPoIB Question

[Michael Krause]

At 10:00 PM 10/23/2006, Greg Lindahl wrote:
>On Mon, Oct 23, 2006 at 07:53:06AM -0500, Hubbell, Sean C 
>Contractor/Decibel wrote:
>
> >   I currently have several applications that uses a legacy IPv4 protocol
> > and I use IPoIB to utilize my infiniband network which works great. I
> > have completed some timing and throughput analysis and noticed that I do
> > not get very much more if I use an infiniband network interface than
> > using my GigE network interface.
>
>You might want to note that different InfinBand implementations have
>quite different performance of IPoIB, especially for UDP.
>
>Another issue is that IPoIB has quite different performance with
>different Linux kernels. This is especially evident for TCP, although
>you can use SDP to accelerate TCP sockets and avoid this issue.
>
> > My question is, am I using IPoIB correctly or are these the typical
> > numbers that everyone is seeing?
>
>It is certainly the case that there are some message patterns and
>situations for which InfiniBand is not much of an improvement over
>gigE.

Unfortunately, the comparison of IB to GbE are often apple-to-orange 
comparisons even for IP over IB.  Until a HCA supplies the same level of 
functional off-load enabled by the IP network stack that is used with 
Ethernet, it really isn't a fair comparison.  The same is also true for 
many of the marketroids and their comparisons of IB to Ethernet based 
solutions.  Fortunately, most customers are getting a bit smarter and not 
falling for the marketing drivel these days - certainly the OEM don't fall 
for it thought the marketroids continue to come in and try to convince 
people it isn't an apple-to-orange comparison.    The fact is both 
technologies have their pros / cons and it is really the workload or 
production environment that determines which is the best fit instead of the 
force fit.

In any case, not really a development issue so will drop further discussion.

Mike