[openib-general] IPoIB breaks

[Roland Dreier]

    Josh> I'm getting great numbers from IPoIB...right up until it
    Josh> dies.  The system is x86_64 and PCIe with latest stuff under
    Josh> 2.6.10.  Streaming tests with both netperf and NetPIPE die
    Josh> almost instantly (NetPIPE actually lasts for a couple
    Josh> iterations).  Ping-pong tests with NetPIPE seem to
    Josh> consistently die when it starts sending message sizes around
    Josh> 2k.

    Josh> Running pings eventually time out with: ping: sendmsg: No
    Josh> buffer space available

    Josh> IPoIB stops working completely until the module is reloaded
    Josh> on both nodes.  Any ideas?

What FW version are you using?  If you are using the latest subversion
sources, then FW 4.6.2 is required (because of the CQ bug in FW
4.6.1).  If you're already on FW 4.6.2, are there any messages in the
kernel log?

(By the way, I've run NPtcp on a dual Nocona/PCIe system overnight and
transferred about 5 TB without a glitch)

 - R.