[ofw] some WinOF questions

[Tzachi Dar]

Diego, did you receive my mail with the attachment?

I just received a mail with "Delivery Notification - RE: [ofw] some
WinOF questions", so I want to be sure you have received it.

Thanks
Tzachi 

> -----Original Message-----
> From: Tzachi Dar 
> Sent: Thursday, November 08, 2007 10:16 AM
> To: 'Diego Guella'; Smith, Stan; Sufficool, Stanley; Hefty, Sean
> Cc: ofw at lists.openfabrics.org
> Subject: RE: [ofw] some WinOF questions
> 
> Hi Diego,
> 
> Please note that there is a bug on the way the buffer sizes 
> between windows and Linux work.
> In short this means that windows starts with big buffers and 
> never changes them. On Linux on the other hand, they start 
> with small buffers and increase them. When windows and Linux 
> work together, this ends with the connection using small 
> buffers and therefore low BW (~3000 Mb/sec).
> 
> There are two workarounds for this problem:
> 
> 1) Changing the code on Linux, please do the following:
> It seems that the root of the "problem" is on line 180 of the 
> file sdp_cma.c There is a function called sdp_init_qp() and 
> there is a line:
> 
> 	sdp_sk(sk)->recv_frags = 0;
> 
> This line tells the Linux side to start with smaller packets. 
> If they change this line to be 
> 
> sdp_sk(sk)->recv_frags = 8; 
> 
> Than Linux will start with bigger packets and (8*4k + 2k 
> =34kb) and they can also get high performance without 
> changing anything at the windows side. Please note that 
> without changing anything at the windows side, the packet 
> size will default to 16kb which will somewhat hurt performance.
> 
> 2) Changing the code on windows:
> Attached is the windows driver that has a work around for the problem.
> It does two things:
> 	1) Increase the windows buffers to 32kb.
> 	2) fools the Linux side to be using large buffers. This 
> should allow the Linux driver to send bigger packets without 
> any changes, and should allow them to reach near wire speed BW.
> To run their test, there is no need to do anything beside 
> copying the file to the windows\system32\drivers directory, 
> and restarting SDP. 
> 
> Two more things to note:
> 1) you can get to a better BW by using windows XP 64 bits. 
> It's price should be similar to the XP 32.
> 2) The fixes that I have sent are meant to increase the BW 
> when Linux is sending to windows. If you want BW to increase 
> on the other side (windows to Linux) you should probably 
> apply both fixes.
> 
> Thanks
> Tzachi
> 
> > -----Original Message-----
> > From: Diego Guella [mailto:diego.guella at sircomtech.com]
> > Sent: Thursday, November 08, 2007 9:39 AM
> > To: Smith, Stan; Tzachi Dar; Sufficool, Stanley; Hefty, Sean
> > Cc: ofw at lists.openfabrics.org
> > Subject: Re: [ofw] some WinOF questions
> > 
> > 
> > ----- Original Message -----
> > >From: "Smith, Stan" <stan.smith at intel.com> Microsoft
> > requires those who
> > >distribute SDP to sign a license with MS concerning IP 
> rights of SDP.
> > >Since WinOF is open-source MS has declined to offer an SDP 
> licensing 
> > >option, therefore SDP is not part of WinOF (Windows
> > OpenFabrics which
> > >is based on the openib-windows src tree).
> > >
> > >On the other hand, Mellanox has signed a licensing 
> agreement with MS 
> > >and therefore distributes SDP as part of their WinIB stack 
> which is 
> > >also based on the openIB-windows src plus Mellanox value add.
> > 
> > Oh, licensing problems. So I'll try with Mellanox WinIB 1.3
> > 
> > >The WinOF answer to missing SDP is WSD (Winsock Direct
> > Provider) which
> > >from the user's perspective provides similar utilization of
> > InfiniBand
> > >hardware as does SDP.
> > 
> > I think we can't use WSD.
> > 1) Requires Windows Server 2003, wich means too many Euros (or $).
> > 2) Can't communicate with Linux using WSD
> > 
> > 
> > >Within the Linux world, connected mode IPoIB approaches or in some 
> > >cases surpasses SDP performance.
> > >Connected mode IPoIB is appearing on the WinOF 2008 horizon.
> > 
> > If it means early 2008, cool! Otherwise I think I need to 
> find another 
> > solution
> > 
> > 
> > 
> > Thanks,
> > Diego
> > 
> >