[ofa-general] AMP/RDMA performance expectation

[yangdong]

Hello! I implemented a message passing subsystem -- AMP for our cluster
file system--DCFS3,
I want AMP to run over Infiniband, I designed it in kernel(verbs).
when client writing file, my protocol is implemented by RDMA WRITE, and
reading file by RDMA
READ, but perf of AMP is less than my expectation. Single thread (1t)
read is 215MB/s, 4t is 340MB/s.
And I see that the read file perf of NFS/RDMA is close to 600MB/s (1t).
I made use of FMR and All physical reg ..and scatter/gather, but why the
perf of AMP is lower?



-- 
Dong Yang <yangdong at ncic.ac.cn>
Institute of Computing Technology, Chinese Academy of Sciences
Address: National Research Center for Intelligent Computing Systems (NCIC), P.O. Box 2704, Beijing 100080, P.R. China
Phone: +86-10-62601005