[openib-general] ip over ib throughtput

[Talpey, Thomas]

At 10:05 AM 5/10/2006, Shirley Ma wrote:
>I meant payload less than or equal to 2044, not IB MTU. IPoIB can only 
>send <=2044 payload per ib_send_post(). NFS/RDMA in this case send 
>32KB per ib_post_send(). 

Actually, in the cases I mentioned earlier, the NFS/RDMA server is
posting 8 4KB RDMA writes and one ~200 byte send to satisfy the
32KB direct read issued by the client. It's possible for the client to
construct many other requests however, so it's possible to result in
a 32KB single inline (nonRDMA) message, or if scatter/gather memory
registration is available, a single 32KB RDMA followed by the 200 byte
reply. Obviously, there are significant resource differences between
these. Which one to use can depend on many factors.

>It would be nice to know the performance 
>difference under same payload for IPoIB over UD and NFS/RDMA. Is that 
>possible? 

Sure, but I wonder why it's interesting. Nobody ever uses NFS in such
small blocksizes, and 2044 bytes would mean, say, 1800 bytes of payload.
What data are you looking for, throughput and overhead? Direct RDMA,
or inline?

Tom.