[openib-general] [Openib-windows] File transfer performance options

[Tzachi Dar]

There is one thing that is missing from your mail, and that is if you
want to see the windows machine as some file server (for example SAMBA,
NFS, SRP), or are you ready to accept it as a normal server. The big
difference is that on the second option the server can be running at
user mode (for example FTP server).

When (the server application is) running at user mode, SDP can be used
as a socket provider.  This means that theoretically every socket
application should run and enjoy the speed of Infiniband. Currently
there are two projects of SDP under development: one is for Linux and
the other for Windows, so SDP can be used to allow machines from both
types to connect. Performance that we have measured on the windows
platform, using DDR cards was bigger than 1200 MB/Sec. (of course, this
data was from host memory, and not from disks).

So, if all you need to do is to pass files from one side to the other, I
would recommend that you will check this option.

One note about your experiments: when using ram disks, this probably
means that there is one more copy from the ram disk to the application
buffer. A real disk, has it's DMA engine, while a ram disk doesn't.
Another copy is probably not a problem when you are talking about
100MB/sec, but it would become a problem once you will use SDP (I hope).

Thanks
Tzachi



	
	
	We've been testing an application that archives large quantities
of data 
	from a Linux system onto a Windows-based server (64bit server
2003 R2).
	
	As part of the investigation into relatively modest transfer
speeds in the 
	win-linux configuration, we configured a Linux-Linux transfer
via IpoIB with 
	NFS layered on top (with ram disks to avoid physical disk
issues)
	
	[Whilst for a real Linux-Linux configuration I would look for
the RDMA over 
	NFS solution, this wouldn't translate to our eventual win-linux 
	inter-operable system.]
	
	I was surprised that even on linux-linux I hit a wall of 100MB/s
(test notes 
	below). Are others doing better? I was hoping for 150MB/s -
200MB/s
	
	Does anyone have any hints on tweaking of an IPoIB/NFS solution
to get 
	better throughput for large files (not so concerned about
latency).
	
	Are there any other inter-operable windows-linux solutions now? 
	(cross-platform NFS over RDMA or SRP initiator/target?)
	
	Paul Baxter