[ofa-general] [PATCHv2] IB/ipoib: S/G and HW checksum support

[Todd Rimmer]

> From: Michael S. Tsirkin
> Sent: Thursday, September 06, 2007 1:12 PM
> To: Jason Gunthorpe
> Cc: Eli Cohen; Michael S. Tsirkin; general at lists.openfabrics.org
> Subject: Re: [ofa-general] [PATCHv2] IB/ipoib: S/G and HW checksum
support
> 
> > >
> > > A (IB) ---- B (Gateway eg HW based) ---- C (Ethernet)
> > >
> > > scheme, in case A does not compute the TCP checksum of a packet,
its
> > > note the role of the gateway to do so, and C would just drop it?!
> >
> > I think the proper way to view Michael's patch, and indeed this
whole
> > idea, is that it just moves the work around, with the goal of
> > eliminating the work for a class of communication (Linux host to
Linux
> > host). So yes, if a gateway uses this feature then it must
regenerate
> > the checksum before it forwards it.
> >
> > It is actually a pretty neat idea, I've never heard of another
network
> > doing this. I wouldn't call it hardware checksum, but more like a
> > peer-to-peer VNIC scheme. Nobody would object if a vnic driver moved
> > checksum and segmentation offload to the VNIC device over a RC QP,
and
> > I think the same rational for that applies here, except it is now
peer
> > to peer. (Michael maybe that is a good name for this concept:
p2p_vnic?)
> 
> Yea. Roland, does the argument sound convincing to you?

I have been observing this discussion and one of my serious concerns in
this scenario is the lack of overlapping "data integrity checking
domains".

In short, enterprise quality networks need to have mechanisms which
prevent an intermediate component from corrupting data which will be
delivered as "valid" to the final destination.

Standard TCP/IP networks address this problem by not having the TCP
checksums recomputed by routers.  Hence if the router corrupts the
packet internally, the IP header may have a valid checksum, however the
TCP checksum would be bad and the final destination would reject the
packet.

IB similarly protects data by having two CRCs (ICRC which is an end to
end CRC, and VCRC which can change per switch/router hop).  Hence a
switch or router problem will result in packets with a bad ICRC which
will be dropped.

Michael's proposal is a nice optimization for the direct host to host
case.

However as soon as a gateway/router (B above) is added there is a
serious gap in the integrity domains.  A hardware problem (or software
bug) in B could undetectably corrupt the packet, but it would be
delivered to C with a valid checksum.  Hence an undetected data
corruption for the overall network path A<->C.

Undetected data corruption is a very nasty word for the enterprise and
designs must strive to remove opportunities for such problems.

Hence I agree with Roland's comment that the name should imply the
serious risk that this option can introduce and it should clearly not be
the default behavior.

Michael's idea of doing this in a manner so the unchecksum'ed packets
are unroutable may also be reasonable.

Todd Rimmer