[ofa-general] Re: Can Mellanox cards handle scatter gather I/O?

[Ralph Campbell]

On Wed, 2007-08-15 at 09:05 +0300, Michael S. Tsirkin wrote:
> > Quoting Kent Baxley <kbaxley at redhat.com>:
> > Subject: Can Mellanox cards handle scatter gather I/O?
> > 
> > 
> > Evidently, we're seeing performance problems when a card on one side
> > is capable of scatter gather IO but the mellanox driver is not able to
> > handle it. Then a whole lot of time is spent gathering the data into
> > one skbuff.
> 
> I guess you are speaking about a router configuration where
> you stick 2 network cards in the same machine - right?
> 
> > This is currently being observed on RHEL4.5 with OFED 1.2.
> > 
> > Is the Mellanox card capable of doing scatter gather IO and this just
> > a driver limitation or is the mellanox card just not able to do scatter
> > gather IO?
> 
> AFAIK all Mellanox cards are capable of s/g IO in hardware.
> 
> However, speaking of IPoIB, the code in
> net/core/dev.c below would just remove NETIF_F_SG immediately:
> 
>         if ((dev->features & NETIF_F_SG) &&
>             !(dev->features & NETIF_F_ALL_CSUM)) {
>                 printk(KERN_NOTICE "%s: Dropping NETIF_F_SG since no checksum feature.\n",
>                        dev->name);
>                 dev->features &= ~NETIF_F_SG;
>         }
> 
> we can work around this code in OFED, but I think it would
> make sense to first have a discussion on netdev and lkml list about why is
> this code there in the first place.

I believe earlier Linux kernels forced HW checksum off unless the device
also supported scatter/gather.  Recent kernels have changed the code
so that the two features can be separately turned on/off.