[ofa-general] IPoIB CM mode and packet drops

[Jason Gunthorpe]

On Fri, Jun 05, 2009 at 11:29:58AM -0400, Hal Rosenstock wrote:

> I've seen some comments about this occuring prior to path MTU
> discovery. Is the issue is that some IP services don't utilize path
> MTU discovery ?

This is the standard problem with mixed MTU segments, it happens on
ethernet too.

path mtu only helps if everything is done properly,  and ICMP Can't
Fragment messages are being produced.

In your case, the IPoIB stack on your host is dropping because it
cannot open an RC connection to the target. (The target is UD only)

So in this case the kernel on the server should be generating ICMP
Can't Fragment errors back to its own networking stack at the same
time it prints that dmesg. No idea if there is code to do this or if
it is happening for you. It is a bug if that code is missing..
tcpdump can probably tell you.

When the server's kernel receives those icmps from itself it can
reduce the MTU to that host.

Jason