Okay, it's late. I have NetPIPE TCP running on ipoib, peak is around 1.9 gigabits/sec or so with no socket buffer tuning. (this means I have an SM and such). Tommorow I will try the netpipe on the uverbs.