<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns:o = "urn:schemas-microsoft-com:office:office" xmlns:st1 =
"urn:schemas-microsoft-com:office:smarttags"><HEAD>
<META content="text/html; charset=iso-8859-1" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.7601.17998"></HEAD>
<BODY>
<DIV><SPAN class=187323909-29112012><FONT face=Arial><FONT size=2>Hi
everyone,</FONT></FONT></SPAN></DIV>
<DIV><FONT size=2 face=Arial><SPAN
class=187323909-29112012></SPAN></FONT> </DIV>
<DIV><FONT size=2 face=Arial><SPAN class=187323909-29112012>Attached please find
a patch for review and discussion (based on rev. 3419 of <A
href="https://beany.openfabrics.org/svnrepo/ofw/gen1/trunk/">https://beany.openfabrics.org/svnrepo/ofw/gen1/trunk/</A>)
which contains a porting of Sean Hefty's RSockets protocol to Windows.
</SPAN></FONT><FONT size=2 face=Arial><SPAN class=187323909-29112012>Many thanks
to Sean at this point for his cooperation and support.</SPAN></FONT></DIV>
<DIV><FONT size=2 face=Arial><SPAN
class=187323909-29112012></SPAN></FONT> </DIV>
<DIV><FONT size=2 face=Arial><SPAN class=187323909-29112012>What I've done is
merging the librdmacm sources from current Linux OFED with the respective OFW
sources, resulting in a librdmacm.dll acting as a Winsock base transport
provider (similar to the ulp/wsd component). In contrast to WSD or ND, that
RSocket<SPAN class=221182909-10012013>s</SPAN> provider allows socket-based RDMA
communication not only between Windows hosts, but also between Windows and
Linux.</SPAN></FONT></DIV>
<DIV><FONT size=2 face=Arial><SPAN
class=187323909-29112012></SPAN></FONT> </DIV>
<DIV><FONT size=2 face=Arial><SPAN class=187323909-29112012>For further comments
and documentation please look at ulp\librdmacm\RSocket.txt.</SPAN></FONT></DIV>
<DIV><FONT size=2 face=Arial><SPAN class=187323909-29112012>
<DIV><SPAN class=187323909-29112012><FONT size=2><SPAN
class=282400818-19122012><FONT
color=#0000ff></FONT></SPAN></FONT></SPAN> </DIV>
<DIV><SPAN class=187323909-29112012><FONT size=2><SPAN
class=282400818-19122012><FONT color=#0000ff><FONT color=#000000>The
contribution of the patch is in the name and on behalf of Océ Printing Systems
GmbH. The use of the patch is based in principle upon the terms of the BSD as
noted in ulp\librdmacm\RSocket.txt.</FONT></FONT></SPAN></FONT></SPAN></DIV>
<DIV><SPAN class=187323909-29112012><FONT size=2><SPAN
class=282400818-19122012></SPAN></FONT></SPAN><SPAN
class=187323909-29112012><FONT color=#0000ff size=2><SPAN
class=282400818-19122012></SPAN></FONT></SPAN> </DIV></SPAN></FONT></DIV>
<DIV><FONT size=2 face=Arial><SPAN class=187323909-29112012>Thanks and
Regards,</SPAN></FONT></DIV>
<DIV><FONT size=2 face=Arial><SPAN
class=187323909-29112012>Hubert</SPAN></FONT></DIV>
<DIV><FONT size=2 face=Arial><SPAN
class=187323909-29112012></SPAN></FONT> </DIV>
<DIV><FONT size=2 face=Arial><SPAN
class=187323909-29112012>===================================================================</SPAN></FONT></DIV>
<DIV><FONT size=2 face=Arial><SPAN
class=187323909-29112012><BR> </DIV></SPAN></FONT><FONT size=2
face=Arial><SPAN class=187323909-29112012>
<DIV>
<DIV align=left>Index:
etc/user/gtod.c<BR>===================================================================<BR>---
etc/user/gtod.c (revision 3419)<BR>+++ etc/user/gtod.c (working
copy)<BR>@@ -57,11 +57,12 @@<BR> ptv->tv_usec = (long) (ll -
((LONGLONG)(ptv->tv_sec) * 10000000)) / 10;<BR> }<BR> <BR>-<BR>-//
static __inline<BR>-int gettimeofday(struct timeval *ptv, void
*ignored)<BR>+static /*__inline*/ int gettimeofday(struct timeval *ptv, void
*ignored)<BR> {<BR> static int QueryCounter =
2;<BR>+ static LARGE_INTEGER Frequency = {10000000,0}; /* prevent division
by 0 */<BR>+ static LARGE_INTEGER Offset; /* counter offset for right
time*/<BR>+ static LARGE_INTEGER LastCounter;<BR> FILETIME
CurrentTime;<BR> UNREFERENCED_PARAMETER(ignored);
<BR> <BR>@@ -69,9 +70,6
@@<BR> <BR> if(QueryCounter)<BR> {<BR>- static
LARGE_INTEGER Frequency;<BR>- static LARGE_INTEGER Offset; /* counter
offset for right time*/<BR>- static LARGE_INTEGER
LastCounter;<BR> LARGE_INTEGER
Time;<BR> LARGE_INTEGER Counter;<BR> <BR>@@ -80,13
+78,15 @@<BR> <BR> if(QueryCounter ==
2)<BR> {<BR>- QueryCounter =
1;<BR>- if(!QueryPerformanceFrequency(&Frequency))<BR>+ if(QueryPerformanceFrequency(&Frequency))<BR> {<BR>+ QueryCounter
= 1; // But now the Frequency is
valid!<BR>+ }<BR>+ else<BR>+ {<BR> QueryCounter
= 0;<BR> Frequency.QuadPart = 10000000; /* prevent
division by 0
*/<BR> }<BR>- <BR> /* get
time as a large integer */<BR> Counter.HighPart &=
0x7FL; /* Clear high bits to prevent overflow
*/<BR> Offset.LowPart =
CurrentTime.dwLowDateTime;<BR>Index:
inc/user/linux/_fcntl.h<BR>===================================================================<BR>---
inc/user/linux/_fcntl.h (revision 0)<BR>+++
inc/user/linux/_fcntl.h (working copy)<BR>@@ -0,0 +1,47 @@<BR>+/*<BR>+ *
Copyright (c) 2012 Oce Printing Systems GmbH. All rights reserved.<BR>+
*<BR>+ * This software is available to you under the BSD license below:<BR>+
*<BR>+ * Redistribution and use in source and binary
forms, with or<BR>+ * without modification, are
permitted provided that the following<BR>+ * conditions
are met:<BR>+ *<BR>+ * - Redistributions of source
code must retain the above<BR>+ *
copyright notice, this list of conditions and the following<BR>+
* disclaimer.<BR>+ *<BR>+
* - Redistributions in binary form must reproduce
the above<BR>+ * copyright notice,
this list of conditions and the following<BR>+
* disclaimer in the documentation
and/or other materials<BR>+ * provided
with the distribution.<BR>+ *<BR>+ * - Neither the
name Oce Printing Systems GmbH nor the names<BR>+
* of the authors may be used to
endorse or promote products<BR>+ *
derived from this software without specific prior written<BR>+
* permission.<BR>+ *<BR>+ * THIS
SOFTWARE IS PROVIDED “AS IS” AND ANY EXPRESS OR IMPLIED<BR>+ * WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE WARRANTIES OF<BR>+ * MERCHANTABILITY AND
FITNESS FOR A PARTICULAR PURPOSE AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN
NO EVENT SHALL THE AUTHORS<BR>+ * OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE
FOR ANY DIRECT,<BR>+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF<BR>+
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;<BR>+ * OR
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF<BR>+ * LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+ * (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE. <BR>+ */<BR>+<BR>+#ifndef
__FCNTL_H_<BR>+#define __FCNTL_H_<BR>+<BR>+#include
<fcntl.h><BR>+<BR>+#define
F_GETFL
3 /* get file->f_flags */<BR>+#define
F_SETFL
4 /* set file->f_flags
*/<BR>+<BR>+#endif /* __FCNTL_H_ */<BR>Index:
tools/dirs<BR>===================================================================<BR>---
tools/dirs (revision 3419)<BR>+++ tools/dirs (working copy)<BR>@@ -6,4
+6,5 @@<BR> part_man \<BR> infiniband-diags
\<BR> qlgcvnic_config \<BR>- ndinstall<BR>+ ndinstall
\<BR>+ rsinstall<BR>Index:
tools/rsinstall/dirs<BR>===================================================================<BR>---
tools/rsinstall/dirs (revision 0)<BR>+++ tools/rsinstall/dirs (working
copy)<BR>@@ -0,0 +1,2 @@<BR>+DIRS=\<BR>+ user<BR>Index:
tools/rsinstall/user/makefile<BR>===================================================================<BR>---
tools/rsinstall/user/makefile (revision 0)<BR>+++
tools/rsinstall/user/makefile (working copy)<BR>@@ -0,0 +1,8 @@<BR>+#<BR>+#
DO NOT EDIT THIS FILE!!! Edit .\sources. if you want to add a new
source<BR>+# file to this component. This file merely indirects to the
real make file<BR>+# that is shared by all the driver components of the OpenIB
Windows project.<BR>+#<BR>+MINIMUM_NT_TARGET_VERSION=0x502<BR>+<BR>+!INCLUDE
..\..\..\inc\openib.def<BR>Index:
tools/rsinstall/user/rsinstall.c<BR>===================================================================<BR>---
tools/rsinstall/user/rsinstall.c (revision 0)<BR>+++
tools/rsinstall/user/rsinstall.c (working copy)<BR>@@ -0,0 +1,724
@@<BR>+/*<BR>+ * Copyright (c) 2005 SilverStorm Technologies. All rights
reserved.<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All rights
reserved.<BR>+ *<BR>+ * This software is available to you under the BSD license
below:<BR>+ *<BR>+ * Redistribution and use in source
and binary forms, with or<BR>+ * without modification,
are permitted provided that the following<BR>+ *
conditions are met:<BR>+ *<BR>+ * -
Redistributions of source code must retain the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer.<BR>+ *<BR>+ * - Redistributions in
binary form must reproduce the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer in the documentation and/or other materials<BR>+
* provided with the distribution.<BR>+
*<BR>+ * - Neither the name Oce Printing Systems
GmbH nor the names<BR>+ * of the
authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR>+ */<BR>+<BR>+/*<BR>+ * Module Name: rsinstall.c<BR>+
* Description: This module installs/removes a winsock service provider for
infiniband. <BR>+ * execute:<BR>+ * To install the service
provider<BR>+ * installsp -i <BR>+ * To
remove the service provider<BR>+ * installsp -r<BR>+
*/<BR>+<BR>+#include "rdma/rwinsock.h"<BR>+#include
<ws2spi.h><BR>+#include <stdio.h><BR>+#include
<stdlib.h><BR>+<BR>+/* Initialize the LSP's provider path for Infiband
Service Provider dll */<BR>+static const WCHAR provider_path[] =
L"%SYSTEMROOT%\\system32\\librdmacm.dll";<BR>+static const WCHAR
provider_prefix[] =L" RSockets for InfiniBand"; //includes one
whitespace<BR>+static const char provider_name[] = VER_PROVIDER
;//L"%VER_PROVIDER% RSockets for InfiniBand"; //(VER_PROVIDER ##
WINDIR);<BR>+static const char openib_key_name[] =
IB_COMPANYNAME;<BR>+<BR>+#ifdef PERFMON_ENABLED<BR>+#include
"LoadPerf.h"<BR>+#include "rdma/rs_regpath.h"<BR>+<BR>+typedef struct
_pm_symbol_def<BR>+{<BR>+ DWORD name_def;<BR>+ CHAR name_str[40];<BR>+ CHAR name_desc[40];<BR>+ CHAR help_desc[256];<BR>+<BR>+}
pm_symbol_def_t;<BR>+<BR>+static pm_symbol_def_t
_pm_symbols[]=<BR>+{<BR>+ { RS_PM_OBJ,<BR>+ "RS_PM_OBJ",<BR>+ "IB
RSockets",<BR>+ "InfiniBand RSockets Provider."<BR>+ },<BR>+ {
RS_PM_COUNTER(BYTES_SEND),<BR>+ "RS_PM_BYTES_TX_SEC",<BR>+ "Send
bytes/sec",<BR>+ "Send bytes/second, excluding RDMA
Write."<BR>+ },<BR>+ {
RS_PM_COUNTER(BYTES_RECV),<BR>+ "RS_PM_BYTES_RX_SEC",<BR>+ "Recv
bytes/sec",<BR>+ "Receive bytes/second, excluding RDMA
Read."<BR>+ },<BR>+ {
RS_PM_COUNTER(BYTES_WRITE),<BR>+ "RS_PM_RDMA_WR_SEC",<BR>+ "RDMA Write
bytes/sec",<BR>+ "RDMA Write bytes/second."<BR>+ },<BR>+ {
RS_PM_COUNTER(BYTES_READ),<BR>+ "RS_PM_RDMA_RD_SEC",<BR>+ "RDMA Read
bytes/sec",<BR>+ "RDMA Read bytes/second."<BR>+ },<BR>+ {
RS_PM_COUNTER(BYTES_TOTAL),<BR>+ "RS_PM_BYTES_SEC",<BR>+ "Total
bytes/sec",<BR>+ "Total bytes transmitted per second, including send,
"<BR>+ "receive, RDMA Write, and RDMA Read."<BR>+ },<BR>+ {
RS_PM_COUNTER(COMP_SEND),<BR>+ "RS_PM_SEND_COMPLETIONS_SEC",<BR>+ "Send
Completions/sec",<BR>+ "Send and RDMA Write
Completions/sec."<BR>+ },<BR>+ {
RS_PM_COUNTER(COMP_RECV),<BR>+ "RS_PM_RECV_COMPLETIONS_SEC",<BR>+ "Recv
Completions/sec",<BR>+ "Recv and RDMA Read
Completions/sec."<BR>+ },<BR>+ {
RS_PM_COUNTER(COMP_TOTAL),<BR>+ "RS_PM_COMPLETIONS_SEC",<BR>+ "Total
Completions/sec",<BR>+ "Total Completions processed per
second."<BR>+ },<BR>+ {
RS_PM_COUNTER(INTR_TOTAL),<BR>+ "RS_PM_COMPLETIONS_INTR",<BR>+ "Total
Interrupts/sec",<BR>+ "Completion Queue events per
second."<BR>+ }<BR>+};<BR>+<BR>+#define RS_PM_NUM_SYMBOLS
(sizeof(_pm_symbols)/sizeof(pm_symbol_def_t))<BR>+#define RS_PM_LANGUAGE "009"
/* good for English */<BR>+<BR>+static CHAR *<BR>+_RSGenerateFileName(char
*header, char *file )<BR>+{<BR>+ DWORD size1, size;<BR>+ CHAR
*full_file_name;<BR>+ int header_len = header == NULL ? 0 :
strlen(header);<BR>+<BR>+ size = GetTempPath(0, NULL);<BR>+ if (size
== 0)<BR>+ {<BR>+ fprintf( stderr, "GetTempPath failed\n"
);<BR>+ return NULL;<BR>+ }<BR>+ size1 = size +
strlen(file) + header_len;<BR>+ full_file_name = HeapAlloc (GetProcessHeap
(), HEAP_ZERO_MEMORY, size1);<BR>+ if ( full_file_name == NULL
)<BR>+ {<BR>+ fprintf( stderr, "GetTempPath failed\n"
);<BR>+ return NULL;<BR>+ }<BR>+ size1 = GetTempPath(size,
full_file_name + header_len);<BR>+ if (size != size1 + 1)
<BR>+ {<BR>+ fprintf( stderr, "Very strange, GetTempPath
returned something different\n" );<BR>+ HeapFree (GetProcessHeap (),
0, full_file_name);<BR>+ return NULL;<BR>+ }<BR>+ if
(header_len != 0)<BR>+ {<BR>+ memcpy(full_file_name, header,
header_len);<BR>+ }<BR>+ strcat(full_file_name,
file);<BR>+ return full_file_name;<BR>+}<BR>+<BR>+<BR>+static
DWORD<BR>+_RSPerfmonIniFilesGenerate( void
)<BR>+{<BR>+ FILE *f_handle;<BR>+ DWORD num;<BR>+ DWORD
ret = ERROR_SUCCESS;<BR>+ char *ibsp_pm_sym_file = NULL;<BR>+ char
*ibsp_pm_ini_file = NULL;<BR>+<BR>+ /* create ".h" file first
*/<BR>+ ibsp_pm_sym_file = _RSGenerateFileName(NULL,
RS_PM_SYM_H_FILE);<BR>+ if( !ibsp_pm_sym_file
)<BR>+ {<BR>+ fprintf( stderr, "_RSGenerateFileName
failed\n" );<BR>+ ret = ERROR_NOT_ENOUGH_MEMORY;<BR>+ goto
Cleanup;<BR>+ }<BR>+<BR>+ f_handle = fopen( ibsp_pm_sym_file, "w+"
);<BR>+<BR>+ if( !f_handle )<BR>+ {<BR>+ fprintf( stderr,
"Create Header file %s failed\n", ibsp_pm_sym_file );<BR>+ ret =
ERROR_FILE_INVALID;<BR>+ goto
Cleanup;<BR>+ }<BR>+<BR>+ fprintf(<BR>+ f_handle, "/* %s
Generated by program */ \r\n", ibsp_pm_sym_file
);<BR>+ <BR>+ <BR>+ for( num = 0; num <
RS_PM_NUM_SYMBOLS; num++ )<BR>+ {<BR>+ fprintf( f_handle,
"#define\t%s\t%d\r\n",<BR>+ _pm_symbols[num].name_str,
_pm_symbols[num].name_def );<BR>+ }<BR>+<BR>+ fflush( f_handle
);<BR>+ fclose( f_handle );<BR>+<BR>+ /* create 'ini' file next
*/<BR>+ ibsp_pm_ini_file = _RSGenerateFileName(NULL,
RS_PM_INI_FILE);<BR>+ if( !ibsp_pm_sym_file
)<BR>+ {<BR>+ fprintf( stderr, "_RSGenerateFileName
failed\n" );<BR>+ ret = ERROR_NOT_ENOUGH_MEMORY;<BR>+ goto
Cleanup;<BR>+ }<BR>+ f_handle = fopen( ibsp_pm_ini_file, "w+"
);<BR>+<BR>+ if( !f_handle )<BR>+ {<BR>+ fprintf( stderr,
"Create INI file %s failed\n", ibsp_pm_ini_file );<BR>+ ret =
ERROR_FILE_INVALID;<BR>+ goto
Cleanup;<BR>+ }<BR>+ <BR>+ fprintf( f_handle,
"[info]\r\ndrivername="
RS_PM_SUBKEY_NAME<BR>+ "\r\nsymbolfile=%s\r\n\r\n", ibsp_pm_sym_file
);<BR>+ fprintf( f_handle,"[languages]\r\n"
RS_PM_LANGUAGE<BR>+ "=language" RS_PM_LANGUAGE "\r\n\r\n"
);<BR>+<BR>+ fprintf( f_handle, <BR>+ "[objects]\r\n%s_"
RS_PM_LANGUAGE
"_NAME=%s\r\n\r\n[text]\r\n",<BR>+ _pm_symbols[0].name_str,
_pm_symbols[0].name_desc );<BR>+ <BR>+ for( num = 0; num <
RS_PM_NUM_SYMBOLS; num++ )<BR>+ {<BR>+ fprintf( f_handle,"%s_"
RS_PM_LANGUAGE "_NAME=%s\r\n",<BR>+ _pm_symbols[num].name_str,
_pm_symbols[num].name_desc );<BR>+ fprintf( f_handle,"%s_"
RS_PM_LANGUAGE "_HELP=%s\r\n",<BR>+ _pm_symbols[num].name_str,
_pm_symbols[num].help_desc );<BR>+ }<BR>+<BR>+ fflush( f_handle
);<BR>+ fclose( f_handle );<BR>+<BR>+Cleanup:<BR>+ if (
ibsp_pm_sym_file )<BR>+ {<BR>+ HeapFree (GetProcessHeap (), 0,
ibsp_pm_sym_file);<BR>+ }<BR>+ if ( ibsp_pm_ini_file
)<BR>+ {<BR>+ HeapFree (GetProcessHeap (), 0,
ibsp_pm_ini_file);<BR>+ }<BR>+ return ret;<BR>+}<BR>+<BR>+<BR>+static
void<BR>+_RSPerfmonIniFilesRemove( void )<BR>+{<BR>+ char *ibsp_pm_sym_file
= NULL;<BR>+ char *ibsp_pm_ini_file = NULL;<BR>+<BR>+ ibsp_pm_sym_file
= _RSGenerateFileName(NULL, RS_PM_SYM_H_FILE);<BR>+ if( !ibsp_pm_sym_file
)<BR>+ {<BR>+ fprintf( stderr, "_RSGenerateFileName
failed\n" );<BR>+ goto
Cleanup;<BR>+ }<BR>+<BR>+ ibsp_pm_ini_file = _RSGenerateFileName(NULL,
RS_PM_INI_FILE);<BR>+ if( !ibsp_pm_sym_file
)<BR>+ {<BR>+ fprintf( stderr, "_RSGenerateFileName
failed\n" );<BR>+ goto Cleanup;<BR>+ }<BR>+<BR>+ if(
!DeleteFile( ibsp_pm_ini_file ) )<BR>+ {<BR>+ fprintf( stderr,
"Delete file %s failed status %d\n",<BR>+ ibsp_pm_ini_file,
GetLastError() );<BR>+ }<BR>+ if( !DeleteFile( ibsp_pm_sym_file )
)<BR>+ {<BR>+ fprintf( stderr,"Delete file %s failed status
%d\n",<BR>+ ibsp_pm_sym_file, GetLastError()
);<BR>+ }<BR>+<BR>+Cleanup: <BR>+ if ( ibsp_pm_sym_file
)<BR>+ {<BR>+ HeapFree (GetProcessHeap (), 0,
ibsp_pm_sym_file);<BR>+ }<BR>+ if ( ibsp_pm_ini_file
)<BR>+ {<BR>+ HeapFree (GetProcessHeap (), 0,
ibsp_pm_ini_file);<BR>+ }<BR>+ <BR>+}<BR>+<BR>+<BR>+/* Try to create
Performance Register Keys */<BR>+static LONG<BR>+_RSPerfmonRegisterKeys( void
)<BR>+{<BR>+ LONG reg_status;<BR>+ HKEY pm_hkey;<BR>+ DWORD typesSupp
= 7;<BR>+<BR>+ reg_status = RegCreateKeyEx(
HKEY_LOCAL_MACHINE,<BR>+ RS_PM_REGISTRY_PATH RS_PM_SUBKEY_PERF, 0,
NULL,<BR>+ REG_OPTION_NON_VOLATILE, KEY_ALL_ACCESS, NULL,
&pm_hkey, NULL );<BR>+<BR>+ if( reg_status != ERROR_SUCCESS
)<BR>+ {<BR>+ fprintf(
stderr,<BR>+ "_RSPerfmonRegisterKeys Create Key %s failed with
%d\n",<BR>+ RS_PM_REGISTRY_PATH RS_PM_SUBKEY_PERF, reg_status
);<BR>+ return reg_status;<BR>+ }<BR>+<BR>+ /*
create/assign values to the key */<BR>+ RegSetValueExW( pm_hkey,
L"Library", 0, REG_EXPAND_SZ,<BR>+ (LPBYTE)provider_path,
sizeof(provider_path) );<BR>+<BR>+ RegSetValueEx( pm_hkey, TEXT("Open"), 0,
REG_SZ,<BR>+ (LPBYTE)TEXT("RSPmOpen"), sizeof(TEXT("RSPmOpen"))
);<BR>+<BR>+ RegSetValueEx( pm_hkey, TEXT("Collect"), 0,
REG_SZ,<BR>+ (LPBYTE)TEXT("RSPmCollectData"),
sizeof(TEXT("RSPmCollectData")) );<BR>+<BR>+ RegSetValueEx( pm_hkey,
TEXT("Close"), 0, REG_SZ,<BR>+ (LPBYTE)TEXT("RSPmClose"),
sizeof(TEXT("RSPmClose")) );<BR>+<BR>+ RegFlushKey( pm_hkey
);<BR>+ RegCloseKey( pm_hkey );<BR>+<BR>+ reg_status = RegCreateKeyEx(
HKEY_LOCAL_MACHINE,<BR>+ RS_PM_EVENTLOG_PATH, 0,
NULL,<BR>+ REG_OPTION_NON_VOLATILE, KEY_ALL_ACCESS, NULL,
&pm_hkey, NULL );<BR>+<BR>+ if( reg_status != ERROR_SUCCESS
)<BR>+ {<BR>+ fprintf(stderr, "Create EventLog Key failed with
%d\n", reg_status );<BR>+ return
reg_status;<BR>+ }<BR>+<BR>+ /* create/assign values to the key
*/<BR>+ RegSetValueExW( pm_hkey, L"EventMessageFile", 0,
REG_EXPAND_SZ,\<BR>+ (LPBYTE)provider_path, sizeof(provider_path)
);<BR>+<BR>+ RegSetValueEx( pm_hkey, TEXT("TypesSupported"), 0,
REG_DWORD,<BR>+ (LPBYTE)&typesSupp, sizeof(typesSupp)
);<BR>+<BR>+ RegFlushKey( pm_hkey );<BR>+ RegCloseKey( pm_hkey
);<BR>+<BR>+ return reg_status;<BR>+}<BR>+<BR>+<BR>+/* Try to destroy
Performance Register Keys */<BR>+static LONG<BR>+_RSPerfmonDeregisterKeys( void
)<BR>+{<BR>+ LONG reg_status;<BR>+<BR>+ reg_status =
RegDeleteKeyEx( HKEY_LOCAL_MACHINE,<BR>+ RS_PM_REGISTRY_PATH
RS_PM_SUBKEY_PERF,<BR>+ (KEY_WOW64_32KEY | KEY_WOW64_64KEY), 0
);<BR>+<BR>+ if( reg_status != ERROR_SUCCESS
)<BR>+ {<BR>+ fprintf(
stderr,<BR>+ "_RSPerfmonRegisterKeys Remove SubKey failed with
%d\n",<BR>+ GetLastError()
);<BR>+ }<BR>+<BR>+ reg_status = RegDeleteKeyEx(
HKEY_LOCAL_MACHINE,<BR>+ RS_PM_REGISTRY_PATH, (KEY_WOW64_32KEY |
KEY_WOW64_64KEY), 0 );<BR>+<BR>+ if( reg_status != ERROR_SUCCESS
)<BR>+ {<BR>+ fprintf(
stderr,<BR>+ "_RSPerfmonRegisterKeys Remove SubKey failed with
%d\n",<BR>+ GetLastError()
);<BR>+ }<BR>+<BR>+ reg_status = RegDeleteKeyEx(
HKEY_LOCAL_MACHINE,<BR>+ RS_PM_EVENTLOG_PATH, (KEY_WOW64_32KEY |
KEY_WOW64_64KEY), 0 );<BR>+<BR>+ if( reg_status != ERROR_SUCCESS
)<BR>+ {<BR>+ fprintf(
stderr,<BR>+ "_RSPerfmonRegisterKeys Remove SubKey failed with
%d\n",<BR>+ GetLastError() );<BR>+ }<BR>+<BR>+ return
reg_status;<BR>+}<BR>+<BR>+<BR>+/*<BR>+ * functions will try to register
performance counters<BR>+ * definitions with PerfMon application.<BR>+ * API
externally called by lodctr.exe/unlodctr.exe utilities.<BR>+ */<BR>+static
DWORD<BR>+_RSPerfmonRegisterCounters( void )<BR>+{<BR>+ DWORD
status;<BR>+ char *ibsp_pm_ini_file = NULL;<BR>+<BR>+ ibsp_pm_ini_file
= _RSGenerateFileName("unused ", RS_PM_INI_FILE);<BR>+ if(
!ibsp_pm_ini_file )<BR>+ {<BR>+ fprintf( stderr,
"_RSGenerateFileName failed\n" );<BR>+ status =
ERROR_NOT_ENOUGH_MEMORY;<BR>+ goto
Cleanup;<BR>+ }<BR>+<BR>+ /*<BR>+ * format commandline string,
as per SDK :<BR>+ * Pointer to a null-terminated string that consists
of one or more <BR>+ * arbitrary letters, a space, and then the name
of the initialization<BR>+ * file.<BR>+ */<BR>+ status =
LoadPerfCounterTextStrings( ibsp_pm_ini_file, TRUE );<BR>+ if( status !=
ERROR_SUCCESS )<BR>+ {<BR>+ status =
GetLastError();<BR>+ fprintf(
stderr,<BR>+ "IBSPPerfmonRegisterCounters install failed status
%d\n", status );<BR>+ }<BR>+Cleanup: <BR>+ if ( ibsp_pm_ini_file
)<BR>+ {<BR>+ HeapFree (GetProcessHeap (), 0,
ibsp_pm_ini_file);<BR>+ }<BR>+<BR>+ return
status;<BR>+}<BR>+<BR>+<BR>+/*<BR>+ * functions will try to unregister
performance counters<BR>+ * definitions with PerfMon application.<BR>+ * API
externally called by lodctr.exe/unlodctr.exe utilities.<BR>+ */<BR>+static
DWORD<BR>+_RSPerfmonDeregisterCounters( void )<BR>+{<BR>+ DWORD
status;<BR>+<BR>+ /*<BR>+ * format commandline string, as per SDK
:<BR>+ * Pointer to a null-terminated string that consists of one or
more <BR>+ * arbitrary letters, a space, and then the name of the
initialization<BR>+ * file.<BR>+ */<BR>+ status =
UnloadPerfCounterTextStrings(<BR>+ TEXT("unused ")
TEXT(RS_PM_SUBKEY_NAME), TRUE );<BR>+ if( status != ERROR_SUCCESS
)<BR>+ {<BR>+ fprintf(
stderr,<BR>+ "IBSPPerfmonDeregisterCounters remove failed
status %d\n",<BR>+ status );<BR>+ }<BR>+ return
status;<BR>+}<BR>+<BR>+#endif /* PERFMON_ENABLED */<BR>+<BR>+<BR>+/*<BR>+ *
Function: usage<BR>+ * Description: Prints usage information.<BR>+
*/<BR>+static void<BR>+usage (char *progname)<BR>+{<BR>+ printf ("usage: %s
[-i/-r [-p]]\n", progname);<BR>+ printf (" -i
[path] Install the service
provider,\n"<BR>+ "
optionally specify full pathname of
DLL\n"<BR>+ "
-r Remove the %s service
provider\n"<BR>+ " -r <name>
Remove the specified service
provider\n"<BR>+ "
-l List service
providers\n",VER_PROVIDER);<BR>+}<BR>+<BR>+<BR>+/* Function:
print_providers<BR>+ * Description: <BR>+ *
This function prints out each entry in the Winsock catalog.<BR>+*/<BR>+static
void print_providers(void)<BR>+{<BR>+ WSAPROTOCOL_INFOW
*protocol_info;<BR>+ unsigned int protocol_count;<BR>+ unsigned int
i;<BR>+ DWORD protocol_size;<BR>+ INT err_no;<BR>+ int
rc;<BR>+<BR>+ /* Find the size of the buffer */<BR>+ protocol_size =
0;<BR>+ rc = WSCEnumProtocols (NULL, NULL, &protocol_size,
&err_no);<BR>+ if (rc == SOCKET_ERROR && err_no != WSAENOBUFS)
{<BR>+ printf("WSCEnumProtocols() returned error (%d)\n",
err_no);<BR>+ return;<BR>+ }<BR>+<BR>+ /* Allocate the
buffer */<BR>+ protocol_info = HeapAlloc (GetProcessHeap (),
HEAP_ZERO_MEMORY, protocol_size);<BR>+ if (protocol_info == NULL)
{<BR>+ printf("HeapAlloc()
failed\n");<BR>+ return;<BR>+ }<BR>+<BR>+ /* Enumerate the
catalog for real */<BR>+ rc = WSCEnumProtocols (NULL, protocol_info,
&protocol_size, &err_no);<BR>+ if (rc == SOCKET_ERROR)
{<BR>+ printf("WSCEnumProtocols returned error for real enumeration
(%d)\n",<BR>+ err_no);<BR>+ HeapFree
(GetProcessHeap (), 0,
protocol_info);<BR>+ return;<BR>+ }<BR>+<BR>+ protocol_count
= rc;<BR>+<BR>+ for (i = 0; i < protocol_count; i++)
{<BR>+ printf ("%010d - %S\n",
protocol_info[i].dwCatalogEntryId,<BR>+ protocol_info[i].szProtocol);<BR>+ }<BR>+<BR>+ HeapFree
(GetProcessHeap (), 0,
protocol_info);<BR>+<BR>+ return;<BR>+}<BR>+<BR>+/*<BR>+ * Function:
install_provider<BR>+ * Description: installs the service
provider<BR>+ */<BR>+static void install_provider(LPWSTR
szProviderPath)<BR>+{<BR>+ int rc;<BR>+ INT err_no;<BR>+ LONG
reg_error;<BR>+ WSAPROTOCOL_INFOW provider;<BR>+ HKEY
hkey;<BR>+ size_t res;<BR>+ size_t
st_len;<BR>+<BR>+ ZeroMemory(&provider,
sizeof(provider));<BR>+ <BR>+ /* Setup the values in PROTOCOL_INFO
*/<BR>+ provider.dwServiceFlags1 = <BR>+ XP1_GUARANTEED_DELIVERY
| <BR>+ XP1_GUARANTEED_ORDER | <BR>+ XP1_MESSAGE_ORIENTED
|<BR>+ XP1_GRACEFUL_CLOSE;<BR>+ provider.dwServiceFlags2 =
0; /* Reserved */<BR>+ provider.dwServiceFlags3 = 0; /* Reserved
*/<BR>+ provider.dwServiceFlags4 = 0; /* Reserved */<BR>+// SAN
provider only: provider.dwProviderFlags =
PFL_HIDDEN;<BR>+ provider.ProviderId = rsProviderGuid; /* Service
Provider ID provided by vendor. */<BR>+ provider.dwCatalogEntryId =
0;<BR>+ provider.ProtocolChain.ChainLen = 1; /* Base Protocol Service
Provider */<BR>+ provider.iVersion = 2; /* don't know what it is
*/<BR>+ provider.iAddressFamily = AF_INET;<BR>+ provider.iMaxSockAddr
= 16;<BR>+ provider.iMinSockAddr = 16;<BR>+ provider.iSocketType =
SOCK_STREAM;<BR>+ provider.iProtocol =
IPPROTO_TCP;<BR>+ provider.iProtocolMaxOffset =
0;<BR>+ provider.iNetworkByteOrder =
BIGENDIAN;<BR>+ provider.iSecurityScheme =
SECURITY_PROTOCOL_NONE;<BR>+ provider.dwMessageSize = 0xFFFFFFFF; /* IB
supports 32-bit lengths for data transfers on RC
*/<BR>+ provider.dwProviderReserved = 0;<BR>+<BR>+ st_len =
strlen(provider_name);<BR>+ rc = mbstowcs(provider.szProtocol,
provider_name, st_len); //do not count \0<BR>+ // We can't use there
mbstowcs_s <BR>+ //rc = mbstowcs_s(&convertedChars,
provider.szProtocol, sizeof(provider_name), provider_name,
);<BR>+ if (rc != st_len)
{<BR>+
printf("<install_provider> Can't convert string %s to
WCHAR\n",provider_name);<BR>+
printf("Converted %d from %d\n", rc, st_len);<BR>+
}<BR>+ wcscpy( provider.szProtocol + st_len,
provider_prefix);<BR>+ wprintf(L"Provider protocol = %s\n",
provider.szProtocol);<BR>+ wprintf(L"Provider path
= %s\n", szProviderPath);<BR>+<BR>+ rc =
WSCInstallProvider(<BR>+ (LPGUID)&rsProviderGuid, szProviderPath,
&provider, 1, &err_no );<BR>+ if( rc == SOCKET_ERROR
)<BR>+ {<BR>+ if( err_no == WSANO_RECOVERY
)<BR>+ printf("The provider is already
installed\n");<BR>+ else<BR>+ printf("install_provider:
WSCInstallProvider failed: %d\n", err_no);<BR>+ }<BR>+}<BR>+<BR>+/*<BR>+ *
Function: remove_provider<BR>+ * Description: removes our
provider.<BR>+ */<BR>+static void remove_provider( const char* const
provider_name )<BR>+{<BR>+ int rc;<BR>+ int err_no;<BR>+ LONG
reg_error;<BR>+ HKEY hkey;<BR>+<BR>+ /* Remove from the catalog
*/<BR>+ rc = WSCDeinstallProvider((LPGUID)&rsProviderGuid,
&err_no);<BR>+ if (rc == SOCKET_ERROR) {<BR>+ printf
("WSCDeinstallProvider failed: %d\n", err_no);<BR>+ }<BR>+<BR>+#if 0 //def
_WIN64<BR>+ /* Remove from the 32-bit catalog too! */<BR>+ rc =
WSCDeinstallProvider32((LPGUID)&rsProviderGuid, &err_no);<BR>+ if
(rc == SOCKET_ERROR) {<BR>+ printf ("WSCDeinstallProvider32 failed:
%d\n", err_no);<BR>+ }<BR>+#endif /* _WIN64 */<BR>+}<BR>+<BR>+/*
Function: main<BR>+ *<BR>+ * Description:<BR>+ * Parse
the command line arguments and call either the install or remove<BR>+
* routine.<BR>+ */<BR>+int __cdecl main (int argc, char
*argv[])<BR>+{<BR>+ WSADATA wsd;<BR>+ size_t rc,
st_len;<BR>+ WCHAR
arg_provider_path[MAX_PATH];<BR>+<BR>+ /* Load Winsock */<BR>+ if
(WSAStartup (MAKEWORD (2, 2), &wsd) != 0) {<BR>+ printf
("InstallSP: Unable to load Winsock: %d\n", GetLastError
());<BR>+ return -1;<BR>+ }<BR>+<BR>+ /* Confirm that the
WinSock DLL supports 2.2. Note that if the<BR>+ * DLL supports versions
greater than 2.2 in addition to 2.2, it<BR>+ * will still return 2.2 in
wVersion since that is the version we<BR>+ * requested. */<BR>+ if
(LOBYTE (wsd.wVersion) != 2 || HIBYTE (wsd.wVersion) != 2)
{<BR>+<BR>+ /* Tell the user that we could not find a usable WinSock
DLL. */<BR>+ WSACleanup
();<BR>+ printf<BR>+ ("InstallSP: Unable to find a
usable version of Winsock DLL\n");<BR>+ return
-1;<BR>+ }<BR>+ if (argc < 2) {<BR>+ usage
(argv[0]);<BR>+ return -1;<BR>+ }<BR>+ if ((strlen
(argv[1]) != 2) && (argv[1][0] != '-')<BR>+ &&
(argv[1][0] != '/')) {<BR>+ usage (argv[0]);<BR>+ return
-1;<BR>+ }<BR>+ switch (tolower (argv[1][1])) {<BR>+<BR>+ case
'i':<BR>+ /* Install the Infiniband Service Provider
*/<BR>+ if( argc == 2
) {<BR>+ wcscpy_s(arg_provider_path,
sizeof(arg_provider_path)/2,
provider_path);<BR>+ }<BR>+ else
{<BR>+ st_len = strlen( argv[2] );<BR>+ if
(st_len >= sizeof(arg_provider_path)/2)<BR>+ st_len =
sizeof(arg_provider_path)/2 - 1;<BR>+ rc =
mbstowcs(arg_provider_path, argv[2], st_len); //do not count
\0<BR>+ arg_provider_path[st_len] =
'\0';<BR>+ // We can't use there mbstowcs_s
<BR>+ if (rc != st_len)
{<BR>+ printf("Can't convert string \"%s\" to WCHAR\n",
argv[2]);<BR>+ printf("Converted %d from %d\n", rc,
st_len);<BR>+ wcscpy_s(arg_provider_path,
sizeof(arg_provider_path)/2,
provider_path);<BR>+ }<BR>+ }<BR>+ install_provider(
arg_provider_path );<BR>+<BR>+#ifdef
PERFMON_ENABLED<BR>+ _RSPerfmonIniFilesGenerate();<BR>+ if
( _RSPerfmonRegisterKeys() == ERROR_SUCCESS
)<BR>+ _RSPerfmonRegisterCounters();<BR>+#endif<BR>+ break;<BR>+<BR>+ case
'r':<BR>+ /* Remove the service provider */<BR>+ if( argc
== 2 )<BR>+ remove_provider( openib_key_name
);<BR>+ else<BR>+ remove_provider( argv[2]
);<BR>+#ifdef
PERFMON_ENABLED<BR>+ _RSPerfmonIniFilesRemove();<BR>+ if (
_RSPerfmonDeregisterCounters() == ERROR_SUCCESS
)<BR>+ _RSPerfmonDeregisterKeys();<BR>+#endif<BR>+ break;<BR>+<BR>+ case
'l':<BR>+ /* List existing providers
*/<BR>+ print_providers();<BR>+ break;<BR>+ <BR>+ default:<BR>+ usage
(argv[0]);<BR>+ break;<BR>+ }<BR>+<BR>+ WSACleanup
();<BR>+<BR>+ return 0;<BR>+}<BR>Index:
tools/rsinstall/user/rsinstall.exe.manifest<BR>===================================================================<BR>---
tools/rsinstall/user/rsinstall.exe.manifest (revision 0)<BR>+++
tools/rsinstall/user/rsinstall.exe.manifest (working copy)<BR>@@ -0,0 +1,10
@@<BR>+<?xml version="1.0" encoding="UTF-8"
standalone="yes"?><BR>+ <assembly
xmlns="urn:schemas-microsoft-com:asm.v1"
manifestVersion="1.0"><BR>+ <trustInfo
xmlns="urn:schemas-microsoft-com:asm.v3"><BR>+
<security><BR>+
<requestedPrivileges><BR>+
<requestedExecutionLevel level="requireAdministrator"
uiAccess="false"/><BR>+
</requestedPrivileges><BR>+
</security><BR>+
</trustInfo><BR>+</assembly> <BR>\ No newline at end of
file<BR>Index:
tools/rsinstall/user/rsinstall.rc<BR>===================================================================<BR>---
tools/rsinstall/user/rsinstall.rc (revision 0)<BR>+++
tools/rsinstall/user/rsinstall.rc (working copy)<BR>@@ -0,0 +1,45
@@<BR>+/*<BR>+ * Copyright (c) 2005 SilverStorm Technologies. All rights
reserved.<BR>+ *<BR>+ * This software is available to you under the OpenIB.org
BSD license<BR>+ * below:<BR>+ *<BR>+ * Redistribution
and use in source and binary forms, with or<BR>+ *
without modification, are permitted provided that the following<BR>+
* conditions are met:<BR>+ *<BR>+
* - Redistributions of source code must retain the
above<BR>+ * copyright notice, this
list of conditions and the following<BR>+
* disclaimer.<BR>+ *<BR>+
* - Redistributions in binary form must reproduce
the above<BR>+ * copyright notice,
this list of conditions and the following<BR>+
* disclaimer in the documentation
and/or other materials<BR>+ * provided
with the distribution.<BR>+ *<BR>+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT
WARRANTY OF ANY KIND,<BR>+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO
THE WARRANTIES OF<BR>+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS<BR>+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN<BR>+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN<BR>+
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<BR>+ *
SOFTWARE.<BR>+ */<BR>+<BR>+<BR>+#include <oib_ver.h><BR>+<BR>+#define
VER_FILETYPE VFT_APP<BR>+#define
VER_FILESUBTYPE VFT2_UNKNOWN<BR>+<BR>+#ifdef
DBG<BR>+#define VER_FILEDESCRIPTION_STR "RSockets for InfiniBand
installer (Debug)"<BR>+#else<BR>+#define
VER_FILEDESCRIPTION_STR "RSockets for InfiniBand
installer"<BR>+#endif<BR>+<BR>+#define
VER_INTERNALNAME_STR "rsinstall.exe"<BR>+#define
VER_ORIGINALFILENAME_STR "rsinstall.exe"<BR>+<BR>+#include
<common.ver><BR>Index:
tools/rsinstall/user/RsInstall.sln<BR>===================================================================<BR>---
tools/rsinstall/user/RsInstall.sln (revision 0)<BR>+++
tools/rsinstall/user/RsInstall.sln (working copy)<BR>@@ -0,0 +1,21
@@<BR>+Microsoft Visual Studio Solution File, Format Version
8.00<BR>+Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "InstallSP",
"InstallSP.vcproj",
"{B3A2B7A0-1906-413E-A457-8AD2FC5E88BB}"<BR>+ ProjectSection(ProjectDependencies)
=
postProject<BR>+ EndProjectSection<BR>+EndProject<BR>+Global<BR>+ GlobalSection(SolutionConfiguration)
= preSolution<BR>+ Debug = Debug<BR>+ Release =
Release<BR>+ EndGlobalSection<BR>+ GlobalSection(ProjectConfiguration)
=
postSolution<BR>+ {B3A2B7A0-1906-413E-A457-8AD2FC5E88BB}.Debug.ActiveCfg
=
Debug|Win32<BR>+ {B3A2B7A0-1906-413E-A457-8AD2FC5E88BB}.Debug.Build.0
=
Debug|Win32<BR>+ {B3A2B7A0-1906-413E-A457-8AD2FC5E88BB}.Release.ActiveCfg
=
Release|Win32<BR>+ {B3A2B7A0-1906-413E-A457-8AD2FC5E88BB}.Release.Build.0
=
Release|Win32<BR>+ EndGlobalSection<BR>+ GlobalSection(ExtensibilityGlobals)
=
postSolution<BR>+ EndGlobalSection<BR>+ GlobalSection(ExtensibilityAddIns)
= postSolution<BR>+ EndGlobalSection<BR>+EndGlobal<BR>Index:
tools/rsinstall/user/SOURCES<BR>===================================================================<BR>---
tools/rsinstall/user/SOURCES (revision 0)<BR>+++
tools/rsinstall/user/SOURCES (working copy)<BR>@@ -0,0 +1,30
@@<BR>+TARGETNAME=rsinstall<BR>+TARGETPATH=..\..\..\bin\user\obj$(BUILD_ALT_DIR)<BR>+TARGETTYPE=PROGRAM<BR>+UMTYPE=console<BR>+USE_MSVCRT=1<BR>+<BR>+INCLUDES=..\..\..\inc;\<BR>+ ..\..\..\inc\user;\<BR>+ ..\..\..\ulp\librdmacm\include;\<BR>+ $(PLATFORM_SDK_PATH)\include;<BR>+<BR>+SOURCES=
\<BR>+ rsinstall.rc
\<BR>+ rsinstall.c<BR>+<BR>+USER_C_FLAGS=$(USER_C_FLAGS)<BR>+#
-DPERFMON_ENABLED<BR>+<BR>+TARGETLIBS=\<BR>+ $(SDK_LIB_PATH)\ws2_32.lib
\<BR>+ $(SDK_LIB_PATH)\LoadPerf.lib <BR>+<BR>+MSC_WARNING_LEVEL=
/W3<BR>+<BR>+LINKER_FLAGS=$(LINKER_FLAGS)<BR>+<BR>+SXS_APPLICATION_MANIFEST=rsinstall.exe.manifest<BR>+SXS_ASSEMBLY_VERSION=1.0<BR>+SXS_ASSEMBLY_NAME=rsinstall.exe<BR>+SXS_ASSEMBLY_LANGUAGE=0000<BR>Index:
ulp/librdmacm/AUTHORS<BR>===================================================================<BR>---
ulp/librdmacm/AUTHORS (revision 3419)<BR>+++
ulp/librdmacm/AUTHORS (working copy)<BR>@@ -1 +1,2 @@<BR> Sean
Hefty <<A
href="mailto:sean.hefty@intel.com">sean.hefty@intel.com</A>><BR>+Hubert
Schmitt <<A
href="mailto:hubert.schmitt@oce.com">hubert.schmitt@oce.com</A>><BR>Index:
ulp/librdmacm/COPYING<BR>===================================================================<BR>---
ulp/librdmacm/COPYING (revision 3419)<BR>+++
ulp/librdmacm/COPYING (working copy)<BR>@@ -1,11 +1,11
@@<BR> Copyright (c) 2008 Intel Corporation. All rights
reserved.<BR>+Copyright (c) 2012 Oce Printing Systems GmbH. All rights
reserved.<BR> <BR>-This software is available to you under the
OpenFabrics.org BSD license<BR>-below:<BR>+This software is available to you
under the BSD license below:<BR> <BR>-
Redistribution and use in source and binary forms, with
or<BR>- without modification, are permitted provided
that the following<BR>- conditions are
met:<BR>+ Redistribution and use in source and binary forms,
with or<BR>+ without modification, are permitted provided that
the following<BR>+ conditions are
met:<BR> <BR> - Redistributions of
source code must retain the
above<BR> copyright notice, this
list of conditions and the following<BR>@@ -16,11 +16,21
@@<BR> disclaimer in the
documentation and/or other
materials<BR> provided with the
distribution.<BR> <BR>-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY
OF ANY KIND,<BR>-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF<BR>-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
AWV<BR>-NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS<BR>-BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN<BR>-ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN<BR>-CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE<BR>-SOFTWARE.<BR>+ - Neither the name Oce
Printing Systems GmbH nor the
names<BR>+ of the authors may be used
to endorse or promote products<BR>+
derived from this software without specific prior
written<BR>+
permission.<BR>+<BR>+ THIS SOFTWARE IS PROVIDED “AS IS”
AND ANY EXPRESS OR IMPLIED<BR>+ WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE WARRANTIES OF<BR>+ MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE AND<BR>+ NON-INFRINGEMENT ARE
DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ OR CONTRIBUTOR
OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF<BR>+ SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;<BR>+ OR
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
OF<BR>+ LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
TORT<BR>+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY
WAY OUT OF<BR>+ THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
THE POSSIBILITY<BR>+ OF SUCH DAMAGE. <BR>Index:
ulp/librdmacm/examples/rstream/makefile<BR>===================================================================<BR>---
ulp/librdmacm/examples/rstream/makefile (revision 0)<BR>+++
ulp/librdmacm/examples/rstream/makefile (working copy)<BR>@@ -0,0 +1,7
@@<BR>+#<BR>+# DO NOT EDIT THIS FILE!!! Edit .\sources. if you want to add
a new source<BR>+# file to this component. This file merely indirects to
the real make file<BR>+# that is shared by all the driver components of the
OpenIB Windows project.<BR>+#<BR>+<BR>+!INCLUDE
..\..\..\..\inc\openib.def<BR>Index:
ulp/librdmacm/examples/rstream/makefile.inc<BR>===================================================================<BR>---
ulp/librdmacm/examples/rstream/makefile.inc (revision 0)<BR>+++
ulp/librdmacm/examples/rstream/makefile.inc (working copy)<BR>@@ -0,0 +1,8
@@<BR>+Custom_target:<BR>+!if "$(BUILD_PASS)" == "PASS2" || "$(BUILD_PASS)" ==
"ALL"<BR>+<BR>+!endif<BR>+<BR>+<BR>+<BR>+!INCLUDE
..\..\..\..\inc\mod_ver.def<BR>Index:
ulp/librdmacm/examples/rstream/rstream.c<BR>===================================================================<BR>---
ulp/librdmacm/examples/rstream/rstream.c (revision 0)<BR>+++
ulp/librdmacm/examples/rstream/rstream.c (working copy)<BR>@@ -0,0 +1,749
@@<BR>+/*<BR>+ * Copyright (c) 2011-2012 Intel Corporation. All rights
reserved.<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All rights
reserved.<BR>+ *<BR>+ * This software is available to you under the BSD license
below:<BR>+ *<BR>+ * Redistribution and use in source
and binary forms, with or<BR>+ * without modification,
are permitted provided that the following<BR>+ *
conditions are met:<BR>+ *<BR>+ * -
Redistributions of source code must retain the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer.<BR>+ *<BR>+ * - Redistributions in
binary form must reproduce the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer in the documentation and/or other materials<BR>+
* provided with the distribution.<BR>+
*<BR>+ * - Neither the name Oce Printing Systems
GmbH nor the names<BR>+ * of the
authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR>+ */<BR>+ <BR>+#include <stdio.h><BR>+#include
<stdlib.h><BR>+#include <string.h><BR>+#include
<_errno.h><BR>+#include "../../../../etc/user/gtod.c" //
getimeofday()<BR>+#include "getopt.c"<BR>+#include
<sys/types.h><BR>+#include <sys/time.h><BR>+#include
<netdb.h><BR>+#include <_fcntl.h><BR>+#include
<unistd.h><BR>+<BR>+#include "..\src\openib_osd.h"<BR>+#include
<rdma/rdma_cma.h><BR>+#include <rdma/rwinsock.h><BR>+<BR>+#define
MSG_DONTWAIT 0x80<BR>+<BR>+struct test_size_param {<BR>+ int
size;<BR>+ int option;<BR>+};<BR>+<BR>+static struct test_size_param
test_size[] = {<BR>+ { 1 << 6, 0 },<BR>+ { 1
<< 7, 1 }, { (1 << 7) + (1 << 6),
1},<BR>+ { 1 << 8, 1 }, { (1 << 8) + (1
<< 7), 1},<BR>+ { 1 << 9, 1 }, { (1 <<
9) + (1 << 8), 1},<BR>+ { 1 << 10, 1 }, { (1 << 10)
+ (1 << 9), 1},<BR>+ { 1 << 11, 1 }, { (1 << 11) +
(1 << 10), 1},<BR>+ { 1 << 12, 0 }, { (1 << 12) + (1
<< 11), 1},<BR>+ { 1 << 13, 1 }, { (1 << 13) + (1
<< 12), 1},<BR>+ { 1 << 14, 1 }, { (1 << 14) + (1
<< 13), 1},<BR>+ { 1 << 15, 1 }, { (1 << 15) + (1
<< 14), 1},<BR>+ { 1 << 16, 0 }, { (1 << 16) + (1
<< 15), 1},<BR>+ { 1 << 17, 1 }, { (1 << 17) + (1
<< 16), 1},<BR>+ { 1 << 18, 1 }, { (1 << 18) + (1
<< 17), 1},<BR>+ { 1 << 19, 1 }, { (1 << 19) + (1
<< 18), 1},<BR>+ { 1 << 20, 0 }, { (1 << 20) + (1
<< 19), 1},<BR>+ { 1 << 21, 1 }, { (1 << 21) + (1
<< 20), 1},<BR>+ { 1 << 22, 1 }, { (1 << 22) + (1
<< 21), 1},<BR>+};<BR>+#define TEST_CNT (sizeof test_size / sizeof
test_size[0])<BR>+<BR>+enum rs_optimization
{<BR>+ opt_mixed,<BR>+ opt_latency,<BR>+ opt_bandwidth<BR>+};<BR>+<BR>+static
int rs, lrs;<BR>+static int use_rs = 1;<BR>+static int use_async = 0;<BR>+static
int verify = 0;<BR>+static int flags = 0; //MSG_DONTWAIT;<BR>+static int
poll_timeout = 0;<BR>+static int custom;<BR>+static enum rs_optimization
optimization;<BR>+static int size_option;<BR>+static int iterations =
1;<BR>+static int transfer_size = 1000;<BR>+static int transfer_count =
1000;<BR>+static int buffer_size;<BR>+static char test_name[10] =
"custom";<BR>+static char *port = "7471";<BR>+static char *dst_addr;<BR>+static
char *src_addr;<BR>+static struct timeval start, end;<BR>+static void
*buf;<BR>+<BR>+#define
rs_socket(f,t,p) use_rs ?
WSASocket(f,t,p,rsGetProtocolInfo(NULL),0,0) : socket(f,t,p)<BR>+#define
rs_bind(s,a,l)
bind(s,a,l)<BR>+#define
rs_listen(s,b)
listen(s,b)<BR>+#define
rs_connect(s,a,l)
connect(s,a,l)<BR>+#define
rs_accept(s,a,l)
accept(s,a,l)<BR>+#define
rs_shutdown(s,h)
shutdown(s,h)<BR>+#define
rs_close(s)
closesocket(s)<BR>+#define
rs_recv(s,b,l,f)
recv(s,b,l,f)<BR>+#define
rs_send(s,b,l,f)
send(s,b,l,f)<BR>+#define rs_recvfrom(s,b,l,f,a,al)
recvfrom(s,b,l,f,a,al)<BR>+#define rs_sendto(s,b,l,f,a,al)
sendto(s,b,l,f,a,al)<BR>+#define rs_select(n,rf,wf,ef,t)
select(n,rf,wf,ef,t)<BR>+#define rs_ioctlsocket(s,c,p)
ioctlsocket(s,c,p)<BR>+#define rs_setsockopt(s,l,n,v,ol)
setsockopt(s,l,n,v,ol)<BR>+#define rs_getsockopt(s,l,n,v,ol)
getsockopt(s,l,n,v,ol)<BR>+<BR>+static void size_str(char *str, size_t ssize,
long long size)<BR>+{<BR>+ long long base, fraction = 0;<BR>+ char
mag;<BR>+<BR>+ if (size >= (1 << 30)) {<BR>+ base = 1
<< 30;<BR>+ mag = 'g';<BR>+ } else if (size >= (1
<< 20)) {<BR>+ base = 1 << 20;<BR>+ mag =
'm';<BR>+ } else if (size >= (1 << 10)) {<BR>+ base = 1
<< 10;<BR>+ mag = 'k';<BR>+ } else {<BR>+ base
= 1;<BR>+ mag = '\0';<BR>+ }<BR>+<BR>+ if (size / base <
10)<BR>+ fraction = (size % base) * 10 / base;<BR>+ if
(fraction) {<BR>+ _snprintf(str, ssize, "%lld.%lld%c", size / base,
fraction, mag);<BR>+ } else {<BR>+ _snprintf(str, ssize,
"%lld%c", size / base, mag);<BR>+ }<BR>+}<BR>+<BR>+static void cnt_str(char
*str, size_t ssize, long long cnt)<BR>+{<BR>+ if (cnt >=
1000000000)<BR>+ _snprintf(str, ssize, "%lldb", cnt /
1000000000);<BR>+ else if (cnt >=
1000000)<BR>+ _snprintf(str, ssize, "%lldm", cnt /
1000000);<BR>+ else if (cnt >= 1000)<BR>+ _snprintf(str,
ssize, "%lldk", cnt / 1000);<BR>+ else<BR>+ _snprintf(str,
ssize, "%lld", cnt);<BR>+}<BR>+<BR>+static void
show_perf(void)<BR>+{<BR>+ char str[32];<BR>+ float
usec;<BR>+ long long bytes;<BR>+<BR>+ usec = (float)((end.tv_sec -
start.tv_sec) * 1000000 + (end.tv_usec - start.tv_usec));<BR>+ bytes =
(long long) iterations * transfer_count * transfer_size * 2;<BR>+<BR>+ /*
name size transfers iterations bytes seconds Gb/sec usec/xfer
*/<BR>+ printf("%-10s", test_name);<BR>+ size_str(str, sizeof str,
transfer_size);<BR>+ printf("%-8s", str);<BR>+ cnt_str(str, sizeof
str, transfer_count);<BR>+ printf("%-8s", str);<BR>+ cnt_str(str,
sizeof str, iterations);<BR>+ printf("%-8s", str);<BR>+ size_str(str,
sizeof str, bytes);<BR>+ printf("%-8s",
str);<BR>+ printf("%8.2fs%10.2f%11.2f\n",<BR>+ usec / 1000000.,
(bytes * 8) / (1000. * usec),<BR>+ (usec / iterations) /
(transfer_count * 2));<BR>+}<BR>+<BR>+static int size_to_count(int
size)<BR>+{<BR>+ if (size >= 1000000)<BR>+ return
100;<BR>+ else if (size >= 100000)<BR>+ return
1000;<BR>+ else if (size >= 10000)<BR>+ return
10000;<BR>+ else if (size >= 1000)<BR>+ return
100000;<BR>+ else<BR>+ return 1000000;<BR>+}<BR>+<BR>+static
void init_latency_test(int size)<BR>+{<BR>+ char
sstr[5];<BR>+<BR>+ size_str(sstr, sizeof sstr,
size);<BR>+ _snprintf(test_name, sizeof test_name, "%s_lat",
sstr);<BR>+ transfer_count = 1;<BR>+ transfer_size =
size;<BR>+ iterations = size_to_count(transfer_size);<BR>+}<BR>+<BR>+static
void init_bandwidth_test(int size)<BR>+{<BR>+ char
sstr[5];<BR>+<BR>+ size_str(sstr, sizeof sstr,
size);<BR>+ _snprintf(test_name, sizeof test_name, "%s_bw",
sstr);<BR>+ iterations = 1;<BR>+ transfer_size =
size;<BR>+ transfer_count =
size_to_count(transfer_size);<BR>+}<BR>+<BR>+static void format_buf(void *buf,
int size)<BR>+{<BR>+ uint8_t *array = buf;<BR>+ static uint8_t
data;<BR>+ int i;<BR>+<BR>+ for (i = 0; i < size;
i++)<BR>+ array[i] = data++;<BR>+}<BR>+<BR>+static int
verify_buf(void *buf, int size)<BR>+{<BR>+ static long long
total_bytes;<BR>+ uint8_t *array = buf;<BR>+ static uint8_t
data;<BR>+ int i;<BR>+<BR>+ for (i = 0; i < size; i++,
total_bytes++) {<BR>+ if (array[i] != data++)
{<BR>+ printf("data verification failed byte %lld\n",
total_bytes);<BR>+ return
-1;<BR>+ }<BR>+ }<BR>+ return 0;<BR>+}<BR>+<BR>+static int
do_poll(struct pollfd
*fds)<BR>+{<BR>+ int ret;<BR>+ int nfds =
0;<BR>+ fd_set readfds, writefds, exceptfds;<BR>+ struct timeval
timeout;<BR>+ <BR>+ FD_ZERO(&readfds);<BR>+ FD_ZERO(&writefds);<BR>+ FD_ZERO(&exceptfds);<BR>+ <BR>+ if
(fds->events & (POLLIN | POLLHUP)) {<BR>+ FD_SET(fds->fd,
&readfds);<BR>+ nfds++;<BR>+ }<BR>+ <BR>+ if
(fds->events & POLLOUT) {<BR>+ FD_SET(fds->fd,
&writefds);<BR>+ nfds++;<BR>+ }<BR>+<BR>+ if
(fds->events & ~(POLLIN | POLLOUT)) {<BR>+ FD_SET(fds->fd,
&exceptfds);<BR>+ nfds++;<BR>+ }<BR>+<BR>+ timeout.tv_sec
= poll_timeout / 1000;<BR>+ timeout.tv_usec = timeout.tv_sec ? 0 :
poll_timeout * 1000;<BR>+ <BR>+ do { <BR>+ ret =
rs_select(<BR>+ nfds,<BR>+ FD_ISSET(fds->fd,
&readfds ) ? &readfds :
NULL,<BR>+ FD_ISSET(fds->fd, &writefds ) ?
&writefds : NULL,<BR>+ FD_ISSET(fds->fd,
&exceptfds) ? &exceptfds : NULL,<BR>+ poll_timeout <
0 ? NULL : &timeout<BR>+ );<BR>+ } while
(!ret);<BR>+ <BR>+ return ret == 1 ? 0 : ret;<BR>+}<BR>+<BR>+static
int send_xfer(int size)<BR>+{<BR>+ struct pollfd fds;<BR>+ int offset,
ret;<BR>+ <BR>+ if (verify)<BR>+ format_buf(buf,
size);<BR>+<BR>+ if (use_async) {<BR>+ fds.fd =
rs;<BR>+ fds.events = POLLOUT;<BR>+ }<BR>+<BR>+ for (offset
= 0; offset < size; ) {<BR>+ if (use_async)
{<BR>+ ret = do_poll(&fds);<BR>+ if
(ret)<BR>+ return
ret;<BR>+ }<BR>+<BR>+ ret = (int)rs_send(rs, (char *)buf +
offset, size - offset, flags);<BR>+ if (ret > 0)
{<BR>+ offset += ret;<BR>+ } else if (errno !=
EWOULDBLOCK && errno != EAGAIN)
{<BR>+ perror("rsend");<BR>+ return
ret;<BR>+ }<BR>+ }<BR>+ return 0;<BR>+}<BR>+<BR>+static int
recv_xfer(int size)<BR>+{<BR>+ struct pollfd fds;<BR>+ int offset,
ret;<BR>+ <BR>+ if (use_async) {<BR>+ fds.fd =
rs;<BR>+ fds.events = POLLIN;<BR>+ }<BR>+<BR>+ for (offset
= 0; offset < size; ) {<BR>+ if (use_async)
{<BR>+ ret = do_poll(&fds);<BR>+ if
(ret)<BR>+ return
ret;<BR>+ }<BR>+<BR>+ ret = (int)rs_recv(rs, (char *)buf +
offset, size - offset, flags);<BR>+ if (ret > 0)
{<BR>+ offset += ret;<BR>+ } else if (errno !=
EWOULDBLOCK && errno != EAGAIN)
{<BR>+ perror("rrecv");<BR>+ return
ret;<BR>+ }<BR>+ }<BR>+<BR>+ if (verify)
{<BR>+ ret = verify_buf(buf, size);<BR>+ if
(ret)<BR>+ return ret;<BR>+ }<BR>+ return
0;<BR>+}<BR>+<BR>+static int sync_test(void)<BR>+{<BR>+ int
ret;<BR>+<BR>+ ret = dst_addr ? send_xfer(16) : recv_xfer(16);<BR>+ if
(ret)<BR>+ return ret;<BR>+<BR>+ return dst_addr ? recv_xfer(16)
: send_xfer(16);<BR>+}<BR>+<BR>+static int run_test(void)<BR>+{<BR>+ int
ret, i, t;<BR>+<BR>+ ret = sync_test();<BR>+ if
(ret)<BR>+ goto out;<BR>+<BR>+ gettimeofday(&start,
NULL);<BR>+ for (i = 0; i < iterations; i++) {<BR>+ for (t =
0; t < transfer_count; t++) {<BR>+ ret = dst_addr ?
send_xfer(transfer_size) :<BR>+
recv_xfer(transfer_size);<BR>+ if
(ret)<BR>+ goto
out;<BR>+ }<BR>+ for (t = 0; t < transfer_count; t++)
{<BR>+ ret = dst_addr ? recv_xfer(transfer_size)
:<BR>+
send_xfer(transfer_size);<BR>+ if
(ret)<BR>+ goto
out;<BR>+ }<BR>+ }<BR>+ gettimeofday(&end,
NULL);<BR>+ show_perf();<BR>+ ret = 0;<BR>+<BR>+out:<BR>+ return
ret;<BR>+}<BR>+<BR>+static void set_options(int rs)<BR>+{<BR>+ int
val;<BR>+<BR>+ if (buffer_size) {<BR>+ rs_setsockopt(rs,
SOL_SOCKET, SO_SNDBUF, (void *)
&buffer_size,<BR>+ sizeof
buffer_size);<BR>+ rs_setsockopt(rs, SOL_SOCKET, SO_RCVBUF, (void *)
&buffer_size,<BR>+ sizeof
buffer_size);<BR>+ } else {<BR>+ val = 1 <<
19;<BR>+ rs_setsockopt(rs, SOL_SOCKET, SO_SNDBUF, (void *) &val,
sizeof val);<BR>+ rs_setsockopt(rs, SOL_SOCKET, SO_RCVBUF, (void *)
&val, sizeof val);<BR>+ }<BR>+<BR>+ val =
1;<BR>+ rs_setsockopt(rs, IPPROTO_TCP, TCP_NODELAY, (void *) &val,
sizeof(val));<BR>+<BR>+ val = 1;<BR>+ if (flags &
MSG_DONTWAIT)<BR>+ rs_ioctlsocket(rs, FIONBIO, (u_long
*)&val);<BR>+<BR>+ if (use_rs) {<BR>+ /* Inline size based
on experimental data */<BR>+ if (optimization == opt_latency)
{<BR>+ val = 384;<BR>+ rs_setsockopt(rs,
SOL_RDMA, RDMA_INLINE, (char *)&val, sizeof val);<BR>+ } else if
(optimization == opt_bandwidth) {<BR>+ val =
0;<BR>+ rs_setsockopt(rs, SOL_RDMA, RDMA_INLINE, (char
*)&val, sizeof val);<BR>+ }<BR>+ }<BR>+}<BR>+<BR>+static int
server_listen(void)<BR>+{<BR>+ struct addrinfo hints, *res;<BR>+ int
val, ret;<BR>+<BR>+ memset(&hints, 0, sizeof
hints);<BR>+ hints.ai_flags = RAI_PASSIVE;<BR>+ hints.ai_family
= AF_INET;<BR>+ hints.ai_socktype =
SOCK_STREAM;<BR>+ hints.ai_protocol = IPPROTO_TCP;<BR>+ <BR>+
ret = getaddrinfo(src_addr, port, &hints, &res);<BR>+ if
(ret) {<BR>+ perror("getaddrinfo");<BR>+ return
ret;<BR>+ }<BR>+<BR>+ lrs = (int)(rs_socket(res->ai_family,
res->ai_socktype, res->ai_protocol));<BR>+ if (lrs < 0)
{<BR>+ perror("rsocket");<BR>+ ret =
lrs;<BR>+ goto free;<BR>+ }<BR>+<BR>+ val =
1;<BR>+ ret = rs_setsockopt(lrs, SOL_SOCKET, SO_REUSEADDR, (char
*)&val, sizeof val);<BR>+ if (ret)
{<BR>+ perror("rsetsockopt SO_REUSEADDR");<BR>+ goto
close;<BR>+ }<BR>+<BR>+ ret = rs_bind(lrs, res->ai_addr,
res->ai_addrlen);<BR>+ if (ret)
{<BR>+ perror("rbind");<BR>+ goto
close;<BR>+ }<BR>+<BR>+ ret = rs_listen(lrs, 1);<BR>+ if
(ret)<BR>+ perror("rlisten");<BR>+<BR>+close:<BR>+ if
(ret)<BR>+ rs_close(lrs);<BR>+free:<BR>+ freeaddrinfo(res);<BR>+ return
ret;<BR>+}<BR>+<BR>+static int server_connect(void)<BR>+{<BR>+ struct
pollfd fds;<BR>+ int ret = 0;<BR>+<BR>+ set_options(lrs);<BR>+ do
{<BR>+ if (use_async) {<BR>+ fds.fd =
lrs;<BR>+ fds.events = POLLIN;<BR>+<BR>+ ret =
do_poll(&fds);<BR>+ if (ret)
{<BR>+ perror("rpoll");<BR>+ return
ret;<BR>+ }<BR>+ }<BR>+<BR>+ rs =
(int)(rs_accept(lrs, NULL, 0));<BR>+ } while (rs < 0 && (errno
== EAGAIN || errno == EWOULDBLOCK));<BR>+<BR>+ if (rs < 0)
{<BR>+ ret =
rs;<BR>+ perror("raccept");<BR>+ return
ret;<BR>+ }<BR>+ set_options(rs);<BR>+<BR>+ return
ret;<BR>+}<BR>+<BR>+static int client_connect(void)<BR>+{<BR>+ struct
addrinfo hints, *res;<BR>+ struct pollfd fds;<BR>+ int ret,
err;<BR>+ socklen_t len;<BR>+ <BR>+ memset(&hints, 0, sizeof
hints);<BR>+ hints.ai_flags = RAI_PASSIVE;<BR>+ hints.ai_family
= AF_INET;<BR>+ hints.ai_socktype =
SOCK_STREAM;<BR>+ hints.ai_protocol = IPPROTO_TCP;<BR>+ <BR>+
ret = getaddrinfo(dst_addr, port, &hints, &res);<BR>+ if
(ret) {<BR>+ perror("getaddrinfo");<BR>+ return
ret;<BR>+ }<BR>+<BR>+ rs = (int)(rs_socket(res->ai_family,
res->ai_socktype, res->ai_protocol));<BR>+ if (rs < 0)
{<BR>+ perror("rsocket");<BR>+ ret =
rs;<BR>+ goto
free;<BR>+ }<BR>+<BR>+ set_options(rs);<BR>+ /* TODO: bind client
to src_addr */<BR>+<BR>+ ret = rs_connect(rs, res->ai_addr,
res->ai_addrlen);<BR>+ if (ret && (errno != EINPROGRESS))
{<BR>+ perror("rconnect");<BR>+ goto
close;<BR>+ }<BR>+<BR>+ if (ret && (errno == EINPROGRESS))
{<BR>+ fds.fd = rs;<BR>+ fds.events =
POLLOUT;<BR>+ ret = do_poll(&fds);<BR>+ if
(ret)<BR>+ goto close;<BR>+<BR>+ len = sizeof
err;<BR>+ ret = rs_getsockopt(rs, SOL_SOCKET, SO_ERROR, (char
*)&err, &len);<BR>+ if (ret)<BR>+ goto
close;<BR>+ if (err) {<BR>+ ret =
-1;<BR>+ errno = err;<BR>+ perror("async
rconnect");<BR>+ }<BR>+ }<BR>+<BR>+close:<BR>+ if
(ret)<BR>+ rs_close(rs);<BR>+free:<BR>+ freeaddrinfo(res);<BR>+ return
ret;<BR>+}<BR>+<BR>+static int run(void)<BR>+{<BR>+ int i, ret =
0;<BR>+ DWORD dwBytesReturned = 0;<BR>+ <BR>+ buf =
malloc(!custom ? test_size[TEST_CNT - 1].size : transfer_size);<BR>+ if
(!buf) {<BR>+ perror("malloc");<BR>+ return
-1;<BR>+ }<BR>+ if (!dst_addr) {<BR>+ ret =
server_listen();<BR>+ if (ret)<BR>+ goto
free;<BR>+ }<BR>+ printf("%-10s%-8s%-8s%-8s%-8s%8s
%10s%13s\n",<BR>+ "name", "bytes",
"xfers", "iters", "total", "time", "Gb/sec", "usec/xfer");<BR>+ if
(!custom) {<BR>+ optimization = opt_latency;<BR>+ ret =
dst_addr ? client_connect() : server_connect();<BR>+ if
(ret)<BR>+ goto free;<BR>+<BR>+ for (i = 0; i <
TEST_CNT; i++) {<BR>+ if (test_size[i].option >
size_option)<BR>+ continue;<BR>+ init_latency_test(test_size[i].size);<BR>+ run_test();<BR>+ }<BR>+ rs_shutdown(rs,
SHUT_RDWR);<BR>+ rs_close(rs);<BR>+<BR>+ optimization =
opt_bandwidth;<BR>+ ret = dst_addr ? client_connect() :
server_connect();<BR>+ if (ret)<BR>+ goto
free;<BR>+<BR>+ for (i = 0; i < TEST_CNT; i++)
{<BR>+ if (test_size[i].option >
size_option)<BR>+ continue;<BR>+ init_bandwidth_test(test_size[i].size);<BR>+ run_test();<BR>+ }<BR>+ }
else {<BR>+ ret = dst_addr ? client_connect() :
server_connect();<BR>+ if (ret)<BR>+ goto
free;<BR>+<BR>+ ret =
run_test();<BR>+ }<BR>+<BR>+ rs_shutdown(rs,
SHUT_RDWR);<BR>+ rs_close(rs);<BR>+free:<BR>+ free(buf);<BR>+ return
ret;<BR>+}<BR>+<BR>+static int set_test_opt(char *optarg)<BR>+{<BR>+ if
(strlen(optarg) == 1) {<BR>+ switch (optarg[0])
{<BR>+ case 's':<BR>+ use_rs =
0;<BR>+ break;<BR>+/* async not yet supported<BR>+
* case 'a':<BR>+ * use_async = 1;<BR>+
* break;<BR>+ */<BR>+ case
'b':<BR>+ flags &=
~MSG_DONTWAIT;<BR>+ break;<BR>+ case
'n':<BR>+ flags |=
MSG_DONTWAIT;<BR>+ break;<BR>+ case
'v':<BR>+ verify =
1;<BR>+ break; default:<BR>+ return
-1;<BR>+ }<BR>+ } else {<BR>+ if
(!_strnicmp("socket", optarg, 6)) {<BR>+ use_rs =
0;<BR>+ } /* async not yet supported<BR>+
* else if (!_strnicmp("async", optarg, 5)) {<BR>+
* use_async = 1;<BR>+ *
}<BR>+ */<BR>+ else if
(!_strnicmp("block", optarg, 5)) {<BR>+ flags &=
~MSG_DONTWAIT;<BR>+ } else if (!_strnicmp("nonblock", optarg, 8))
{<BR>+ flags |= MSG_DONTWAIT;<BR>+ } else if
(!_strnicmp("verify", optarg, 6)) {<BR>+ verify =
1;<BR>+ } else {<BR>+ return
-1;<BR>+ }<BR>+ }<BR>+ return 0;<BR>+}<BR>+<BR>+int
main(int argc, char **argv)<BR>+{<BR>+ int op, ret;<BR>+ WSADATA
wsaData;<BR>+<BR>+ if (0 != (ret =
WSAStartup(0x202,&wsaData)) )
{<BR>+ fprintf(stderr, "WSAStartup
failed with error %d\n",ret);<BR>+ ret =
-1;<BR>+ goto
out;<BR>+ }<BR>+ while ((op = getopt(argc, argv,
"s:b:B:I:C:S:p:T:")) != -1) {<BR>+ switch (op) {<BR>+ case
's':<BR>+ dst_addr =
optarg;<BR>+ break;<BR>+ case
'b':<BR>+ src_addr =
optarg;<BR>+ break;<BR>+ case
'B':<BR>+ buffer_size =
atoi(optarg);<BR>+ break;<BR>+ case
'I':<BR>+ custom = 1;<BR>+ iterations =
atoi(optarg);<BR>+ break;<BR>+ case
'C':<BR>+ custom = 1;<BR>+ transfer_count =
atoi(optarg);<BR>+ break;<BR>+ case
'S':<BR>+ if (!_strnicmp("all", optarg, 3))
{<BR>+ size_option = 1;<BR>+ } else
{<BR>+ custom =
1;<BR>+ transfer_size =
atoi(optarg);<BR>+ }<BR>+ break;<BR>+ case
'p':<BR>+ port =
optarg;<BR>+ break;<BR>+ case
'T':<BR>+ if
(!set_test_opt(optarg))<BR>+ break;<BR>+ /*
invalid option - fall through
*/<BR>+ default:<BR>+ printf("usage: %s\n",
argv[0]);<BR>+ printf("\t[-s
server_address]\n");<BR>+ printf("\t[-b
bind_address]\n");<BR>+ printf("\t[-B
buffer_size]\n");<BR>+ printf("\t[-I
iterations]\n");<BR>+ printf("\t[-C
transfer_count]\n");<BR>+ printf("\t[-S transfer_size or
all]\n");<BR>+ printf("\t[-p
port_number]\n");<BR>+ printf("\t[-T
test_option]\n");<BR>+ printf("\t s|sockets -
use standard tcp/ip
sockets\n");<BR>+// printf("\t a|async -
asynchronous operation (use
poll)\n");<BR>+ printf("\t b|blocking - use
blocking calls\n");<BR>+ printf("\t
n|nonblocking - use nonblocking
calls\n");<BR>+ printf("\t v|verify - verify
data\n");<BR>+ exit(1);<BR>+ }<BR>+ }<BR>+ if
(!(flags & MSG_DONTWAIT))<BR>+ poll_timeout =
-1;<BR>+<BR>+ ret = run();<BR>+<BR>+out:<BR>+
WSACleanup();<BR>+<BR>+ return ret;<BR>+}<BR>Index:
ulp/librdmacm/examples/rstream/rstream.rc<BR>===================================================================<BR>---
ulp/librdmacm/examples/rstream/rstream.rc (revision 0)<BR>+++
ulp/librdmacm/examples/rstream/rstream.rc (working copy)<BR>@@ -0,0 +1,45
@@<BR>+/*<BR>+ * Copyright (c) 2005 Mellanox Technologies. All rights
reserved.<BR>+ *<BR>+ * This software is available to you under the OpenIB.org
BSD license<BR>+ * below:<BR>+ *<BR>+ * Redistribution
and use in source and binary forms, with or<BR>+ *
without modification, are permitted provided that the following<BR>+
* conditions are met:<BR>+ *<BR>+
* - Redistributions of source code must retain the
above<BR>+ * copyright notice, this
list of conditions and the following<BR>+
* disclaimer.<BR>+ *<BR>+
* - Redistributions in binary form must reproduce
the above<BR>+ * copyright notice,
this list of conditions and the following<BR>+
* disclaimer in the documentation
and/or other materials<BR>+ * provided
with the distribution.<BR>+ *<BR>+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT
WARRANTY OF ANY KIND,<BR>+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO
THE WARRANTIES OF<BR>+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS<BR>+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN<BR>+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN<BR>+
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<BR>+ *
SOFTWARE.<BR>+ */<BR>+<BR>+<BR>+#include <oib_ver.h><BR>+<BR>+#define
VER_FILETYPE VFT_APP<BR>+#define
VER_FILESUBTYPE VFT2_UNKNOWN<BR>+<BR>+#ifdef
DBG<BR>+#define VER_FILEDESCRIPTION_STR "(R)Socket Test
(Debug)"<BR>+#else<BR>+#define VER_FILEDESCRIPTION_STR "(R)Socket
Test "<BR>+#endif<BR>+<BR>+#define
VER_INTERNALNAME_STR "rstream.exe"<BR>+#define
VER_ORIGINALFILENAME_STR "rstream.exe"<BR>+<BR>+#include
<common.ver><BR>Index:
ulp/librdmacm/examples/rstream/SOURCES<BR>===================================================================<BR>---
ulp/librdmacm/examples/rstream/SOURCES (revision 0)<BR>+++
ulp/librdmacm/examples/rstream/SOURCES (working copy)<BR>@@ -0,0 +1,32
@@<BR>+TARGETNAME=rstream<BR>+TARGETPATH=..\..\..\..\bin\user\obj$(BUILD_ALT_DIR)<BR>+TARGETTYPE=PROGRAM<BR>+UMTYPE=console<BR>+USE_MSVCRT=1<BR>+NTTARGETFILES=Custom_target<BR>+<BR>+C_DEFINES=$(C_DEFINES)
/D__WIN__ <BR>+<BR>+SOURCES=rstream.rc
\<BR>+ rstream.c<BR>+<BR>+INCLUDES= ..;
\<BR>+ ..\..\..\..\inc;
\<BR>+ ..\..\..\..\inc\user;
\<BR>+ ..\..\..\..\inc\user\linux;
\<BR>+ ..\..\include;
\<BR>+ ..\..\..\libibverbs\include;
\<BR>+ ..\..\..\..\hw\mlx4\user\hca;
\<BR>+ ..\..\..\..\inc\complib;
\<BR>+ ..\..\..\..\core\complib\user\$(O);
\<BR>+ ..\..\..\..\core\al\user\$(O);
\<BR>+ ..\..\..\..\etc\user;<BR>+<BR>+RCOPTIONS=/I..\..\win\include<BR>+<BR>+TARGETLIBS=
\<BR>+ $(DDK_LIB_PATH)\Ws2_32.lib
\<BR>+ $(TARGETPATH)\*\complib.lib
\<BR>+ $(TARGETPATH)\*\ibal.lib<BR>+<BR>+MSC_WARNING_LEVEL=
/W3<BR>Index:
ulp/librdmacm/include/rdma/rdma_cma.h<BR>===================================================================<BR>---
ulp/librdmacm/include/rdma/rdma_cma.h (revision 3419)<BR>+++
ulp/librdmacm/include/rdma/rdma_cma.h (working copy)<BR>@@ -1,8 +1,8
@@<BR> /*<BR> * Copyright (c) 2005-2009 Intel Corporation. All
rights reserved.<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All
rights reserved.<BR> *<BR>- * This software is available to you under the
OpenFabrics.org BSD license<BR>- * below:<BR>+ * This software is available to
you under the BSD license below:<BR> *<BR> *
Redistribution and use in source and binary forms, with or<BR>
* without modification, are permitted provided that the
following<BR>@@ -17,14 +17,24 @@<BR>
* disclaimer in the documentation
and/or other materials<BR> *
provided with the distribution.<BR> *<BR>- * THE SOFTWARE IS PROVIDED "AS
IS", WITHOUT WARRANTY OF ANY KIND,<BR>- * EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO THE WARRANTIES OF<BR>- * MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AWV<BR>- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS<BR>- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN<BR>- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN<BR>-
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<BR>- *
SOFTWARE.<BR>+ * - Neither the name Oce Printing
Systems GmbH nor the names<BR>+ * of
the authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR> */<BR> <BR> #pragma once<BR>@@ -678,7 +688,10
@@<BR> /* Option details
*/<BR> enum<BR> {<BR>- RDMA_OPTION_ID_TOS = 0 /*
uint8_t: RFC 2474 */<BR>+ RDMA_OPTION_ID_TOS = 0, /*
uint8_t: RFC 2474 */<BR>+ RDMA_OPTION_ID_REUSEADDR = 1, /* int:
~SO_REUSEADDR */<BR>+ RDMA_OPTION_ID_AFONLY = 2, /* int:
~IPV6_V6ONLY */<BR>+ RDMA_OPTION_IB_PATH = 1 /* struct
ibv_path_data[] */<BR> };<BR> <BR> /**<BR>Index:
ulp/librdmacm/include/rdma/rdma_verbs.h<BR>===================================================================<BR>---
ulp/librdmacm/include/rdma/rdma_verbs.h (revision 3419)<BR>+++
ulp/librdmacm/include/rdma/rdma_verbs.h (working copy)<BR>@@ -1,297 +1,297
@@<BR>-/*<BR>- * Copyright (c) 2010 Intel Corporation. All rights
reserved.<BR>- *<BR>- * This software is available to you under a choice of one
of two<BR>- * licenses. You may choose to be licensed under the terms of
the GNU<BR>- * General Public License (GPL) Version 2, available from the
file<BR>- * COPYING in the main directory of this source tree, or the<BR>- *
OpenIB.org BSD license below:<BR>- *<BR>- *
Redistribution and use in source and binary forms, with or<BR>-
* without modification, are permitted provided that the
following<BR>- * conditions are met:<BR>- *<BR>-
* - Redistributions of source code must retain the
above<BR>- * copyright notice, this
list of conditions and the following<BR>-
* disclaimer.<BR>- *<BR>-
* - Redistributions in binary form must reproduce
the above<BR>- * copyright notice,
this list of conditions and the following<BR>-
* disclaimer in the documentation
and/or other materials<BR>- * provided
with the distribution.<BR>- *<BR>- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT
WARRANTY OF ANY KIND,<BR>- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO
THE WARRANTIES OF<BR>- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
AND<BR>- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS<BR>- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN<BR>- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN<BR>-
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<BR>- *
SOFTWARE.<BR>- */<BR>-<BR>-#if !defined(RDMA_VERBS_H)<BR>-#define
RDMA_VERBS_H<BR>-<BR>-#include <assert.h><BR>-#include
<infiniband/verbs.h><BR>-#include <rdma/rdma_cma.h><BR>-#include
<errno.h><BR>-<BR>-#ifdef __cplusplus<BR>-extern "C"
{<BR>-#endif<BR>-<BR>-static __inline int rdma_seterrno(int
ret)<BR>-{<BR>- if (ret) {<BR>- errno = ret;<BR>- ret
= -1;<BR>- }<BR>- return ret;<BR>-}<BR>-<BR>-/*<BR>- * Memory
registration helpers.<BR>- */<BR>-static __inline struct ibv_mr
*<BR>-rdma_reg_msgs(struct rdma_cm_id *id, void *addr, size_t
length)<BR>-{<BR>- return ibv_reg_mr(id->qp->pd, addr, length,
IBV_ACCESS_LOCAL_WRITE);<BR>-}<BR>-<BR>-static __inline struct ibv_mr
*<BR>-rdma_reg_read(struct rdma_cm_id *id, void *addr, size_t
length)<BR>-{<BR>- return ibv_reg_mr(id->qp->pd, addr, length,
IBV_ACCESS_LOCAL_WRITE
|<BR>-
IBV_ACCESS_REMOTE_READ);<BR>-}<BR>-<BR>-static __inline struct ibv_mr
*<BR>-rdma_reg_write(struct rdma_cm_id *id, void *addr, size_t
length)<BR>-{<BR>- return ibv_reg_mr(id->qp->pd, addr, length,
IBV_ACCESS_LOCAL_WRITE
|<BR>-
IBV_ACCESS_REMOTE_WRITE);<BR>-}<BR>-<BR>-static __inline
int<BR>-rdma_dereg_mr(struct ibv_mr *mr)<BR>-{<BR>- return
rdma_seterrno(ibv_dereg_mr(mr));<BR>-}<BR>-<BR>-<BR>-/*<BR>- * Vectored send,
receive, and RDMA operations.<BR>- * Support multiple scatter-gather
entries.<BR>- */<BR>-static __inline int<BR>-rdma_post_recvv(struct rdma_cm_id
*id, void *context, struct ibv_sge *sgl,<BR>- int
nsge)<BR>-{<BR>- struct ibv_recv_wr wr, *bad;<BR>-<BR>- wr.wr_id =
(uintptr_t) context;<BR>- wr.next = NULL;<BR>- wr.sg_list =
sgl;<BR>- wr.num_sge = nsge;<BR>-<BR>- return
rdma_seterrno(ibv_post_recv(id->qp, &wr,
&bad));<BR>-}<BR>-<BR>-static __inline int<BR>-rdma_post_sendv(struct
rdma_cm_id *id, void *context, struct ibv_sge *sgl,<BR>- int nsge,
int flags)<BR>-{<BR>- struct ibv_send_wr wr, *bad;<BR>-<BR>- wr.wr_id
= (uintptr_t) context;<BR>- wr.next = NULL;<BR>- wr.sg_list =
sgl;<BR>- wr.num_sge = nsge;<BR>- wr.opcode =
IBV_WR_SEND;<BR>- wr.send_flags = flags;<BR>-<BR>- return
rdma_seterrno(ibv_post_send(id->qp, &wr,
&bad));<BR>-}<BR>-<BR>-static __inline int<BR>-rdma_post_readv(struct
rdma_cm_id *id, void *context, struct ibv_sge *sgl,<BR>- int nsge,
int flags, uint64_t remote_addr, uint32_t rkey)<BR>-{<BR>- struct
ibv_send_wr wr, *bad;<BR>-<BR>- wr.wr_id = (uintptr_t)
context;<BR>- wr.next = NULL;<BR>- wr.sg_list =
sgl;<BR>- wr.num_sge = nsge;<BR>- wr.opcode =
IBV_WR_RDMA_READ;<BR>- wr.send_flags =
flags;<BR>- wr.wr.rdma.remote_addr = remote_addr;<BR>- wr.wr.rdma.rkey
= rkey;<BR>-<BR>- return rdma_seterrno(ibv_post_send(id->qp, &wr,
&bad));<BR>-}<BR>-<BR>-static __inline int<BR>-rdma_post_writev(struct
rdma_cm_id *id, void *context, struct ibv_sge *sgl,<BR>- int nsge,
int flags, uint64_t remote_addr, uint32_t rkey)<BR>-{<BR>- struct
ibv_send_wr wr, *bad;<BR>-<BR>- wr.wr_id = (uintptr_t)
context;<BR>- wr.next = NULL;<BR>- wr.sg_list =
sgl;<BR>- wr.num_sge = nsge;<BR>- wr.opcode =
IBV_WR_RDMA_WRITE;<BR>- wr.send_flags =
flags;<BR>- wr.wr.rdma.remote_addr = remote_addr;<BR>- wr.wr.rdma.rkey
= rkey;<BR>-<BR>- return rdma_seterrno(ibv_post_send(id->qp, &wr,
&bad));<BR>-}<BR>-<BR>-/*<BR>- * Simple send, receive, and RDMA calls.<BR>-
*/<BR>-static __inline int<BR>-rdma_post_recv(struct rdma_cm_id *id, void
*context, void *addr,<BR>- size_t
length, struct ibv_mr *mr)<BR>-{<BR>- struct ibv_sge
sge;<BR>-<BR>- assert((addr >= mr->addr)
&&<BR>- (((uint8_t) addr + length) <=
((uint8_t) mr->addr + mr->length)));<BR>- sge.addr = (uint64_t)
(uintptr_t) addr;<BR>- sge.length = (uint32_t) length;<BR>- sge.lkey =
mr->lkey;<BR>-<BR>- return rdma_post_recvv(id, context, &sge,
1);<BR>-}<BR>-<BR>-static __inline int<BR>-rdma_post_send(struct rdma_cm_id *id,
void *context, void *addr,<BR>- size_t
length, struct ibv_mr *mr, int flags)<BR>-{<BR>- struct ibv_sge
sge;<BR>-<BR>- sge.addr = (uint64_t) (uintptr_t) addr;<BR>- sge.length
= (uint32_t) length;<BR>- sge.lkey = mr ? mr->lkey :
0;<BR>-<BR>- return rdma_post_sendv(id, context, &sge, 1,
flags);<BR>-}<BR>-<BR>-static __inline int<BR>-rdma_post_read(struct rdma_cm_id
*id, void *context, void *addr,<BR>-
size_t length, struct ibv_mr *mr, int
flags,<BR>- uint64_t remote_addr,
uint32_t rkey)<BR>-{<BR>- struct ibv_sge sge;<BR>-<BR>- sge.addr =
(uint64_t) (uintptr_t) addr;<BR>- sge.length = (uint32_t)
length;<BR>- sge.lkey = mr->lkey;<BR>-<BR>- return
rdma_post_readv(id, context, &sge, 1, flags, remote_addr,
rkey);<BR>-}<BR>-<BR>-static __inline int<BR>-rdma_post_write(struct rdma_cm_id
*id, void *context, void *addr,<BR>- size_t length, struct ibv_mr
*mr, int flags,<BR>- uint64_t remote_addr, uint32_t
rkey)<BR>-{<BR>- struct ibv_sge sge;<BR>-<BR>- sge.addr = (uint64_t)
(uintptr_t) addr;<BR>- sge.length = (uint32_t) length;<BR>- sge.lkey =
mr ? mr->lkey : 0;<BR>-<BR>- return rdma_post_writev(id, context,
&sge, 1, flags, remote_addr, rkey);<BR>-}<BR>-<BR>-static __inline
int<BR>-rdma_post_ud_send(struct rdma_cm_id *id, void *context, void
*addr,<BR>- size_t length, struct ibv_mr *mr, int
flags,<BR>- struct ibv_ah *ah, uint32_t
remote_qpn)<BR>-{<BR>- struct ibv_send_wr wr, *bad;<BR>- struct
ibv_sge sge;<BR>-<BR>- sge.addr = (uint64_t) (uintptr_t)
addr;<BR>- sge.length = (uint32_t) length;<BR>- sge.lkey = mr ?
mr->lkey : 0;<BR>-<BR>- wr.wr_id = (uintptr_t)
context;<BR>- wr.next = NULL;<BR>- wr.sg_list =
&sge;<BR>- wr.num_sge = 1;<BR>- wr.opcode =
IBV_WR_SEND;<BR>- wr.send_flags = flags;<BR>- wr.wr.ud.ah =
ah;<BR>- wr.wr.ud.remote_qpn = remote_qpn;<BR>- wr.wr.ud.remote_qkey =
RDMA_UDP_QKEY;<BR>-<BR>- return rdma_seterrno(ibv_post_send(id->qp,
&wr, &bad));<BR>-}<BR>-<BR>-// Comment out until patch to automatically
create CQs<BR>-static __inline int<BR>-rdma_get_send_comp(struct rdma_cm_id *id,
struct ibv_wc *wc)<BR>-{<BR>- struct ibv_cq *cq;<BR>- void
*context;<BR>- int ret;<BR>-<BR>- ret = ibv_poll_cq(id->send_cq, 1,
wc);<BR>- if (ret)<BR>- goto out;<BR>-<BR>- ret =
ibv_req_notify_cq(id->send_cq, 0);<BR>- if (ret)<BR>- return
rdma_seterrno(ret);<BR>-<BR>- while (!(ret = ibv_poll_cq(id->send_cq, 1,
wc))) {<BR>- ret = ibv_get_cq_event(id->send_cq_channel, &cq,
&context);<BR>- if (ret)<BR>- return
rdma_seterrno(ret);<BR>-<BR>- assert(cq == id->send_cq &&
context == id);<BR>- ibv_ack_cq_events(id->send_cq,
1);<BR>- }<BR>-out:<BR>- return (ret < 0) ? rdma_seterrno(ret) :
ret;<BR>-}<BR>-<BR>-static __inline int<BR>-rdma_get_recv_comp(struct rdma_cm_id
*id, struct ibv_wc *wc)<BR>-{<BR>- struct ibv_cq *cq;<BR>- void
*context;<BR>- int ret;<BR>-<BR>- ret = ibv_poll_cq(id->recv_cq, 1,
wc);<BR>- if (ret)<BR>- goto out;<BR>-<BR>- ret =
ibv_req_notify_cq(id->recv_cq, 0);<BR>- if (ret)<BR>- return
rdma_seterrno(ret);<BR>-<BR>- while (!(ret = ibv_poll_cq(id->recv_cq, 1,
wc))) {<BR>- ret = ibv_get_cq_event(id->recv_cq_channel, &cq,
&context);<BR>- if (ret)<BR>- return
rdma_seterrno(ret);<BR>-<BR>- assert(cq == id->recv_cq &&
context == id);<BR>- ibv_ack_cq_events(id->recv_cq,
1);<BR>- }<BR>-out:<BR>- return (ret < 0) ? rdma_seterrno(ret) :
ret;<BR>-}<BR>-<BR>-#ifdef __cplusplus<BR>-}<BR>-#endif<BR>-<BR>-#endif /*
RDMA_CMA_H */<BR>+/*<BR>+ * Copyright (c) 2010 Intel Corporation. All
rights reserved.<BR>+ *<BR>+ * This software is available to you under a choice
of one of two<BR>+ * licenses. You may choose to be licensed under the
terms of the GNU<BR>+ * General Public License (GPL) Version 2, available from
the file<BR>+ * COPYING in the main directory of this source tree, or the<BR>+ *
OpenIB.org BSD license below:<BR>+ *<BR>+ *
Redistribution and use in source and binary forms, with or<BR>+
* without modification, are permitted provided that the
following<BR>+ * conditions are met:<BR>+ *<BR>+
* - Redistributions of source code must retain the
above<BR>+ * copyright notice, this
list of conditions and the following<BR>+
* disclaimer.<BR>+ *<BR>+
* - Redistributions in binary form must reproduce
the above<BR>+ * copyright notice,
this list of conditions and the following<BR>+
* disclaimer in the documentation
and/or other materials<BR>+ * provided
with the distribution.<BR>+ *<BR>+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT
WARRANTY OF ANY KIND,<BR>+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO
THE WARRANTIES OF<BR>+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS<BR>+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN<BR>+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN<BR>+
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<BR>+ *
SOFTWARE.<BR>+ */<BR>+<BR>+#if !defined(RDMA_VERBS_H)<BR>+#define
RDMA_VERBS_H<BR>+<BR>+#include <assert.h><BR>+#include
<infiniband/verbs.h><BR>+#include <rdma/rdma_cma.h><BR>+#include
<errno.h><BR>+<BR>+#ifdef __cplusplus<BR>+extern "C"
{<BR>+#endif<BR>+<BR>+static __inline int rdma_seterrno(int
ret)<BR>+{<BR>+ if (ret) {<BR>+ errno = ret;<BR>+ ret
= -1;<BR>+ }<BR>+ return ret;<BR>+}<BR>+<BR>+/*<BR>+ * Memory
registration helpers.<BR>+ */<BR>+static __inline struct ibv_mr
*<BR>+rdma_reg_msgs(struct rdma_cm_id *id, void *addr, size_t
length)<BR>+{<BR>+ return ibv_reg_mr(id->qp->pd, addr, length,
IBV_ACCESS_LOCAL_WRITE);<BR>+}<BR>+<BR>+static __inline struct ibv_mr
*<BR>+rdma_reg_read(struct rdma_cm_id *id, void *addr, size_t
length)<BR>+{<BR>+ return ibv_reg_mr(id->qp->pd, addr, length,
IBV_ACCESS_LOCAL_WRITE
|<BR>+
IBV_ACCESS_REMOTE_READ);<BR>+}<BR>+<BR>+static __inline struct ibv_mr
*<BR>+rdma_reg_write(struct rdma_cm_id *id, void *addr, size_t
length)<BR>+{<BR>+ return ibv_reg_mr(id->qp->pd, addr, length,
IBV_ACCESS_LOCAL_WRITE
|<BR>+
IBV_ACCESS_REMOTE_WRITE);<BR>+}<BR>+<BR>+static __inline
int<BR>+rdma_dereg_mr(struct ibv_mr *mr)<BR>+{<BR>+ return
rdma_seterrno(ibv_dereg_mr(mr));<BR>+}<BR>+<BR>+<BR>+/*<BR>+ * Vectored send,
receive, and RDMA operations.<BR>+ * Support multiple scatter-gather
entries.<BR>+ */<BR>+static __inline int<BR>+rdma_post_recvv(struct rdma_cm_id
*id, void *context, struct ibv_sge *sgl,<BR>+ int
nsge)<BR>+{<BR>+ struct ibv_recv_wr wr, *bad;<BR>+<BR>+ wr.wr_id =
(uintptr_t) context;<BR>+ wr.next = NULL;<BR>+ wr.sg_list =
sgl;<BR>+ wr.num_sge = nsge;<BR>+<BR>+ return
rdma_seterrno(ibv_post_recv(id->qp, &wr,
&bad));<BR>+}<BR>+<BR>+static __inline int<BR>+rdma_post_sendv(struct
rdma_cm_id *id, void *context, struct ibv_sge *sgl,<BR>+ int nsge,
int flags)<BR>+{<BR>+ struct ibv_send_wr wr, *bad;<BR>+<BR>+ wr.wr_id
= (uintptr_t) context;<BR>+ wr.next = NULL;<BR>+ wr.sg_list =
sgl;<BR>+ wr.num_sge = nsge;<BR>+ wr.opcode =
IBV_WR_SEND;<BR>+ wr.send_flags = flags;<BR>+<BR>+ return
rdma_seterrno(ibv_post_send(id->qp, &wr,
&bad));<BR>+}<BR>+<BR>+static __inline int<BR>+rdma_post_readv(struct
rdma_cm_id *id, void *context, struct ibv_sge *sgl,<BR>+ int nsge,
int flags, uint64_t remote_addr, uint32_t rkey)<BR>+{<BR>+ struct
ibv_send_wr wr, *bad;<BR>+<BR>+ wr.wr_id = (uintptr_t)
context;<BR>+ wr.next = NULL;<BR>+ wr.sg_list =
sgl;<BR>+ wr.num_sge = nsge;<BR>+ wr.opcode =
IBV_WR_RDMA_READ;<BR>+ wr.send_flags =
flags;<BR>+ wr.wr.rdma.remote_addr = remote_addr;<BR>+ wr.wr.rdma.rkey
= rkey;<BR>+<BR>+ return rdma_seterrno(ibv_post_send(id->qp, &wr,
&bad));<BR>+}<BR>+<BR>+static __inline int<BR>+rdma_post_writev(struct
rdma_cm_id *id, void *context, struct ibv_sge *sgl,<BR>+ int nsge,
int flags, uint64_t remote_addr, uint32_t rkey)<BR>+{<BR>+ struct
ibv_send_wr wr, *bad;<BR>+<BR>+ wr.wr_id = (uintptr_t)
context;<BR>+ wr.next = NULL;<BR>+ wr.sg_list =
sgl;<BR>+ wr.num_sge = nsge;<BR>+ wr.opcode =
IBV_WR_RDMA_WRITE;<BR>+ wr.send_flags =
flags;<BR>+ wr.wr.rdma.remote_addr = remote_addr;<BR>+ wr.wr.rdma.rkey
= rkey;<BR>+<BR>+ return rdma_seterrno(ibv_post_send(id->qp, &wr,
&bad));<BR>+}<BR>+<BR>+/*<BR>+ * Simple send, receive, and RDMA calls.<BR>+
*/<BR>+static __inline int<BR>+rdma_post_recv(struct rdma_cm_id *id, void
*context, void *addr,<BR>+ size_t
length, struct ibv_mr *mr)<BR>+{<BR>+ struct ibv_sge
sge;<BR>+<BR>+ assert((addr >= mr->addr)
&&<BR>+ (((uint8_t) addr + length) <=
((uint8_t) mr->addr + mr->length)));<BR>+ sge.addr = (uint64_t)
(uintptr_t) addr;<BR>+ sge.length = (uint32_t) length;<BR>+ sge.lkey =
mr->lkey;<BR>+<BR>+ return rdma_post_recvv(id, context, &sge,
1);<BR>+}<BR>+<BR>+static __inline int<BR>+rdma_post_send(struct rdma_cm_id *id,
void *context, void *addr,<BR>+ size_t
length, struct ibv_mr *mr, int flags)<BR>+{<BR>+ struct ibv_sge
sge;<BR>+<BR>+ sge.addr = (uint64_t) (uintptr_t) addr;<BR>+ sge.length
= (uint32_t) length;<BR>+ sge.lkey = mr ? mr->lkey :
0;<BR>+<BR>+ return rdma_post_sendv(id, context, &sge, 1,
flags);<BR>+}<BR>+<BR>+static __inline int<BR>+rdma_post_read(struct rdma_cm_id
*id, void *context, void *addr,<BR>+
size_t length, struct ibv_mr *mr, int
flags,<BR>+ uint64_t remote_addr,
uint32_t rkey)<BR>+{<BR>+ struct ibv_sge sge;<BR>+<BR>+ sge.addr =
(uint64_t) (uintptr_t) addr;<BR>+ sge.length = (uint32_t)
length;<BR>+ sge.lkey = mr->lkey;<BR>+<BR>+ return
rdma_post_readv(id, context, &sge, 1, flags, remote_addr,
rkey);<BR>+}<BR>+<BR>+static __inline int<BR>+rdma_post_write(struct rdma_cm_id
*id, void *context, void *addr,<BR>+ size_t length, struct ibv_mr
*mr, int flags,<BR>+ uint64_t remote_addr, uint32_t
rkey)<BR>+{<BR>+ struct ibv_sge sge;<BR>+<BR>+ sge.addr = (uint64_t)
(uintptr_t) addr;<BR>+ sge.length = (uint32_t) length;<BR>+ sge.lkey =
mr ? mr->lkey : 0;<BR>+<BR>+ return rdma_post_writev(id, context,
&sge, 1, flags, remote_addr, rkey);<BR>+}<BR>+<BR>+static __inline
int<BR>+rdma_post_ud_send(struct rdma_cm_id *id, void *context, void
*addr,<BR>+ size_t length, struct ibv_mr *mr, int
flags,<BR>+ struct ibv_ah *ah, uint32_t
remote_qpn)<BR>+{<BR>+ struct ibv_send_wr wr, *bad;<BR>+ struct
ibv_sge sge;<BR>+<BR>+ sge.addr = (uint64_t) (uintptr_t)
addr;<BR>+ sge.length = (uint32_t) length;<BR>+ sge.lkey = mr ?
mr->lkey : 0;<BR>+<BR>+ wr.wr_id = (uintptr_t)
context;<BR>+ wr.next = NULL;<BR>+ wr.sg_list =
&sge;<BR>+ wr.num_sge = 1;<BR>+ wr.opcode =
IBV_WR_SEND;<BR>+ wr.send_flags = flags;<BR>+ wr.wr.ud.ah =
ah;<BR>+ wr.wr.ud.remote_qpn = remote_qpn;<BR>+ wr.wr.ud.remote_qkey =
RDMA_UDP_QKEY;<BR>+<BR>+ return rdma_seterrno(ibv_post_send(id->qp,
&wr, &bad));<BR>+}<BR>+<BR>+// Comment out until patch to automatically
create CQs<BR>+static __inline int<BR>+rdma_get_send_comp(struct rdma_cm_id *id,
struct ibv_wc *wc)<BR>+{<BR>+ struct ibv_cq *cq;<BR>+ void
*context;<BR>+ int ret;<BR>+<BR>+ ret = ibv_poll_cq(id->send_cq, 1,
wc);<BR>+ if (ret)<BR>+ goto out;<BR>+<BR>+ ret =
ibv_req_notify_cq(id->send_cq, 0);<BR>+ if (ret)<BR>+ return
rdma_seterrno(ret);<BR>+<BR>+ while (!(ret = ibv_poll_cq(id->send_cq, 1,
wc))) {<BR>+ ret = ibv_get_cq_event(id->send_cq_channel, &cq,
&context);<BR>+ if (ret)<BR>+ return
rdma_seterrno(ret);<BR>+<BR>+ assert(cq == id->send_cq &&
context == id);<BR>+ ibv_ack_cq_events(id->send_cq,
1);<BR>+ }<BR>+out:<BR>+ return (ret < 0) ? rdma_seterrno(ret) :
ret;<BR>+}<BR>+<BR>+static __inline int<BR>+rdma_get_recv_comp(struct rdma_cm_id
*id, struct ibv_wc *wc)<BR>+{<BR>+ struct ibv_cq *cq;<BR>+ void
*context;<BR>+ int ret;<BR>+<BR>+ ret = ibv_poll_cq(id->recv_cq, 1,
wc);<BR>+ if (ret)<BR>+ goto out;<BR>+<BR>+ ret =
ibv_req_notify_cq(id->recv_cq, 0);<BR>+ if (ret)<BR>+ return
rdma_seterrno(ret);<BR>+<BR>+ while (!(ret = ibv_poll_cq(id->recv_cq, 1,
wc))) {<BR>+ ret = ibv_get_cq_event(id->recv_cq_channel, &cq,
&context);<BR>+ if (ret)<BR>+ return
rdma_seterrno(ret);<BR>+<BR>+ assert(cq == id->recv_cq &&
context == id);<BR>+ ibv_ack_cq_events(id->recv_cq,
1);<BR>+ }<BR>+out:<BR>+ return (ret < 0) ? rdma_seterrno(ret) :
ret;<BR>+}<BR>+<BR>+#ifdef __cplusplus<BR>+}<BR>+#endif<BR>+<BR>+#endif /*
RDMA_CMA_H */<BR>Index:
ulp/librdmacm/include/rdma/rs_regpath.h<BR>===================================================================<BR>---
ulp/librdmacm/include/rdma/rs_regpath.h (revision 0)<BR>+++
ulp/librdmacm/include/rdma/rs_regpath.h (working copy)<BR>@@ -0,0 +1,71
@@<BR>+/*<BR>+ * Copyright (c) 2005 SilverStorm Technologies. All rights
reserved.<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All rights
reserved.<BR>+ *<BR>+ * This software is available to you under the BSD license
below:<BR>+ *<BR>+ * Redistribution and use in source
and binary forms, with or<BR>+ * without modification,
are permitted provided that the following<BR>+ *
conditions are met:<BR>+ *<BR>+ * -
Redistributions of source code must retain the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer.<BR>+ *<BR>+ * - Redistributions in
binary form must reproduce the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer in the documentation and/or other materials<BR>+
* provided with the distribution.<BR>+
*<BR>+ * - Neither the name Oce Printing Systems
GmbH nor the names<BR>+ * of the
authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR>+ */<BR>+<BR>+#ifndef _RS_REGPATH_H_<BR>+#define _RS_REGPATH_H_<BR>+<BR>+/*
these definitions are common for installSP and WSD projects */<BR>+#define
RS_PM_REGISTRY_PATH \<BR>+ TEXT("SYSTEM\\CurrentControlSet\\Services\\RSockets\\")<BR>+#define
RS_PM_EVENTLOG_PATH \<BR>+ TEXT("SYSTEM\\CurrentControlSet\\Services\\EventLog\\Application\\RSockets")<BR>+#define
RS_PM_SUBKEY_NAME TEXT("RSockets")<BR>+#define
RS_PM_SUBKEY_PERF TEXT("Performance")<BR>+#define
RS_PM_INI_FILE "rs_perfcounters.ini"<BR>+#define
RS_PM_SYM_H_FILE "rs_perfini.h"<BR>+<BR>+enum
RS_PM_COUNTERS<BR>+{<BR>+ BYTES_SEND =
0,<BR>+ BYTES_RECV,<BR>+ BYTES_WRITE,<BR>+ BYTES_READ,<BR>+ BYTES_TOTAL,<BR>+ COMP_SEND,<BR>+ COMP_RECV,<BR>+ COMP_TOTAL,<BR>+ INTR_TOTAL,<BR>+ RS_PM_NUM_COUNTERS<BR>+};<BR>+<BR>+/*
counter symbol names */<BR>+#define
RS_PM_OBJ 0<BR>+#define RS_PM_COUNTER( X
) ((X + 1) * 2)<BR>+<BR>+#endif /* _RS_REGPATH_H_ */<BR>Index:
ulp/librdmacm/include/rdma/rsocket.h<BR>===================================================================<BR>---
ulp/librdmacm/include/rdma/rsocket.h (revision 0)<BR>+++
ulp/librdmacm/include/rdma/rsocket.h (working copy)<BR>@@ -0,0 +1,134
@@<BR>+/*<BR>+ * Copyright (c) 2011 Intel Corporation. All rights
reserved.<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All rights
reserved.<BR>+ *<BR>+ * This software is available to you under the BSD license
below:<BR>+ *<BR>+ * Redistribution and use in source
and binary forms, with or<BR>+ * without modification,
are permitted provided that the following<BR>+ *
conditions are met:<BR>+ *<BR>+ * -
Redistributions of source code must retain the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer.<BR>+ *<BR>+ * - Redistributions in
binary form must reproduce the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer in the documentation and/or other materials<BR>+
* provided with the distribution.<BR>+
*<BR>+ * - Neither the name Oce Printing Systems
GmbH nor the names<BR>+ * of the
authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR>+ */<BR>+<BR>+#if !defined(RSOCKET_H)<BR>+#define
RSOCKET_H<BR>+<BR>+#include <infiniband/verbs.h><BR>+#include
<rdma/rdma_cma.h><BR>+#include <sys/socket.h><BR>+#include
<errno.h><BR>+#include <ws2spi.h><BR>+<BR>+typedef unsigned int
nfds_t; // Under Linux from poll.h<BR>+<BR>+#ifdef __cplusplus<BR>+extern "C"
{<BR>+#endif<BR>+__declspec(dllexport)<BR>+int rsocket(int domain, int type, int
protocol);<BR>+__declspec(dllexport)<BR>+int rbind(int socket, const struct
sockaddr *addr, socklen_t addrlen);<BR>+__declspec(dllexport)<BR>+int
rlisten(int socket, int backlog);<BR>+__declspec(dllexport)<BR>+int raccept(int
socket, struct sockaddr *addr, socklen_t *addrlen);<BR>+SOCKET WSPAPI
WSPAccept(<BR>+ SOCKET socket,<BR>+ struct
sockaddr *addr,<BR>+ LPINT
addrlen,<BR>+ LPCONDITIONPROC
lpfnCondition,<BR>+ DWORD_PTR
dwCallbackData,<BR>+ LPINT
lpErrno<BR>+);<BR>+__declspec(dllexport)<BR>+int rconnect(int socket, const
struct sockaddr *addr, socklen_t addrlen);<BR>+__declspec(dllexport)<BR>+int
rshutdown(int socket, int how);<BR>+__declspec(dllexport)<BR>+int rclose(int
socket);<BR>+__declspec(dllexport)<BR>+ssize_t rrecv(int socket, void *buf,
size_t len, int flags);<BR>+__declspec(dllexport)<BR>+ssize_t rrecvfrom(int
socket, void *buf, size_t len, int flags,<BR>+ struct sockaddr
*src_addr, socklen_t *addrlen);<BR>+__declspec(dllexport)<BR>+ssize_t
rrecvmsg(int socket, struct msghdr *msg, int
flags);<BR>+__declspec(dllexport)<BR>+ssize_t rsend(int socket, const void *buf,
size_t len, int flags);<BR>+__declspec(dllexport)<BR>+ssize_t rsendto(int
socket, const void *buf, size_t len, int flags,<BR>+ const struct
sockaddr *dest_addr, socklen_t addrlen);<BR>+__declspec(dllexport)<BR>+ssize_t
rsendmsg(int socket, const struct msghdr *msg, int
flags);<BR>+__declspec(dllexport)<BR>+ssize_t rread(int socket, void *buf,
size_t count);<BR>+__declspec(dllexport)<BR>+ssize_t rreadv(int socket, const
struct iovec *iov, int iovcnt);<BR>+__declspec(dllexport)<BR>+ssize_t rwrite(int
socket, const void *buf, size_t count);<BR>+__declspec(dllexport)<BR>+ssize_t
rwritev(int socket, const struct iovec *iov, int
iovcnt);<BR>+__declspec(dllexport)<BR>+int rpoll(struct pollfd *fds, nfds_t
nfds, int timeout);<BR>+__declspec(dllexport)<BR>+int rselect(int nfds, fd_set
*readfds, fd_set *writefds,<BR>+ fd_set *exceptfds,
struct timeval *timeout);<BR>+__declspec(dllexport)<BR>+int rgetpeername(int
socket, struct sockaddr *addr, socklen_t
*addrlen);<BR>+__declspec(dllexport)<BR>+int rgetsockname(int socket, struct
sockaddr *addr, socklen_t *addrlen);<BR>+<BR>+#ifndef SOL_RDMA<BR>+#define
SOL_RDMA 0x10000<BR>+enum
{<BR>+ RDMA_SQSIZE,<BR>+ RDMA_RQSIZE,<BR>+ RDMA_INLINE<BR>+};<BR>+#endif<BR>+<BR>+__declspec(dllexport)<BR>+int
rsetsockopt(int socket, int level, int optname,<BR>+ const void
*optval, socklen_t optlen);<BR>+__declspec(dllexport)<BR>+int rgetsockopt(int
socket, int level, int optname,<BR>+ void *optval, socklen_t
*optlen);<BR>+__declspec(dllexport)<BR>+int rfcntl(int socket, int cmd, ... /*
arg */ );<BR>+__declspec(dllexport)<BR>+int rioctlsocket(int socket, long cmd,
u_long* argp);<BR>+<BR>+int rsGetStatus ( __out LPRS_STATUS *lppStatusBuffer
);<BR>+<BR>+#ifdef __cplusplus<BR>+}<BR>+#endif<BR>+<BR>+#endif /* RSOCKET_H
*/<BR>Index:
ulp/librdmacm/include/rdma/rwinsock.h<BR>===================================================================<BR>---
ulp/librdmacm/include/rdma/rwinsock.h (revision 0)<BR>+++
ulp/librdmacm/include/rdma/rwinsock.h (working copy)<BR>@@ -0,0 +1,132
@@<BR>+/*<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All rights
reserved.<BR>+ *<BR>+ * This software is available to you under the BSD license
below:<BR>+ *<BR>+ * Redistribution and use in source
and binary forms, with or<BR>+ * without modification,
are permitted provided that the following<BR>+ *
conditions are met:<BR>+ *<BR>+ * -
Redistributions of source code must retain the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer.<BR>+ *<BR>+ * - Redistributions in
binary form must reproduce the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer in the documentation and/or other materials<BR>+
* provided with the distribution.<BR>+
*<BR>+ * - Neither the name Oce Printing Systems
GmbH nor the names<BR>+ * of the
authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR>+ */<BR>+<BR>+#if !defined(RWINSOCK_H)<BR>+#define
RWINSOCK_H<BR>+<BR>+#include <winsock2.h><BR>+#include
<ws2tcpip.h><BR>+#include <stdlib.h><BR>+#include
<string.h><BR>+<BR>+static const GUID rsProviderGuid = {
//D478E78B-A803-4a25-A5E4-83BFB7EAF4A7<BR>+ 0xd478e78b,<BR>+ 0xa803,<BR>+ 0x4a25,<BR>+ {
0xa5, 0xe4, 0x83, 0xbf, 0xb7, 0xea, 0xf4, 0xa7 }<BR>+};<BR>+<BR>+static
WSAPROTOCOL_INFO rsProtocolInfo = {0};<BR>+<BR>+/**<BR>+ *
\brief Get RSockets Winsock provider's WSAPROTOCOL_INFO
structure.<BR>+ *<BR>+ * \param lpStatus Pointer to status variable to be
returned. Can be NULL if not required.<BR>+ *<BR>+ *
\return Pointer to the RSockets Winsock provider's
WSAPROTOCOL_INFO structure<BR>+ * (NULL if the
RSockets provider is not found or another error occured).<BR>+ */<BR>+static
LPWSAPROTOCOL_INFO rsGetProtocolInfo (LPINT
lpStatus)<BR>+{<BR>+ int
Status = ERROR_SUCCESS;<BR>+ LPWSAPROTOCOL_INFO
lpProtocolBuffer = NULL;<BR>+ LPWSAPROTOCOL_INFO
lpReturn = NULL;
<BR>+ DWORD
BufferLength =
0;<BR>+ DWORD
i;<BR>+<BR>+ WSAEnumProtocols (NULL, NULL, &BufferLength); // Should
always return the BufferLength<BR>+<BR>+ if (NULL == (lpProtocolBuffer =
(LPWSAPROTOCOL_INFO)malloc (BufferLength)))<BR>+ {<BR>+ Status =
ERROR_NOT_ENOUGH_MEMORY;<BR>+ goto
cleanup;<BR>+ }<BR>+<BR>+ if (SOCKET_ERROR == WSAEnumProtocols (NULL,
lpProtocolBuffer, &BufferLength))<BR>+ {<BR>+ Status =
WSAGetLastError();<BR>+ goto
cleanup;<BR>+ }<BR>+ <BR>+ for (i = 0; i < BufferLength /
sizeof(*lpProtocolBuffer); i++)<BR>+ {<BR>+ if (0 == memcmp
(&lpProtocolBuffer[i].ProviderId, &rsProviderGuid,
sizeof(rsProviderGuid)))<BR>+ {<BR>+ rsProtocolInfo =
lpProtocolBuffer[i];<BR>+ lpReturn =
&rsProtocolInfo;<BR>+ break;<BR>+ }<BR>+ }<BR>+<BR>+ cleanup:<BR>+ if
(lpProtocolBuffer)<BR>+ free (lpProtocolBuffer);<BR>+
<BR>+ if (lpStatus)<BR>+ *lpStatus =
Status;<BR>+<BR>+ return lpReturn;<BR>+}<BR>+<BR>+#ifndef
SOL_RDMA<BR>+#define SOL_RDMA 0x10000 // for getsockopt + setsockopt<BR>+enum
{<BR>+ RDMA_SQSIZE,<BR>+ RDMA_RQSIZE,<BR>+ RDMA_INLINE<BR>+};<BR>+#endif
/* SOL_RDMA */<BR>+<BR>+typedef struct {<BR>+ struct
sockaddr src_addr;<BR>+ struct
sockaddr dst_addr;<BR>+ char state[16];<BR>+}
RS_STATUS, *LPRS_STATUS;<BR>+<BR>+typedef struct {<BR>+ char
szLine[128];<BR>+} RS_TRACE_OUT, *LPRS_TRACE_OUT;<BR>+<BR>+/*<BR>+ * IOCTL code
definition to get RS_STATUS information via WSAIoctl():<BR>+ */<BR>+#define
IOC_VENDOR_OFA 0x0FA0000<BR>+#define SIO_RS_GET_STATUS (IOC_OUT
| IOC_VENDOR | IOC_VENDOR_OFA | 1)<BR>+#define SIO_RS_GET_TRACE (IOC_OUT |
IOC_VENDOR | IOC_VENDOR_OFA | 2)<BR>+<BR>+#endif /* RWINSOCK_H */<BR>Index:
ulp/librdmacm/RSocket.txt<BR>===================================================================<BR>---
ulp/librdmacm/RSocket.txt (revision 0)<BR>+++
ulp/librdmacm/RSocket.txt (working copy)<BR>@@ -0,0 +1,105 @@<BR>+<BR>+
Copyright (c) 2012 Oce Printing Systems GmbH. All rights
reserved.<BR>+<BR>+ This software is available to you under the BSD license
below:<BR>+<BR>+ Redistribution and use in source and
binary forms, with or<BR>+ without modification, are
permitted provided that the following<BR>+ conditions
are met:<BR>+<BR>+ - Redistributions of source
code must retain the above<BR>+
copyright notice, this list of conditions and the
following<BR>+
disclaimer.<BR>+<BR>+ - Redistributions in binary
form must reproduce the above<BR>+
copyright notice, this list of conditions and the
following<BR>+ disclaimer in the
documentation and/or other
materials<BR>+ provided with the
distribution.<BR>+<BR>+ - Neither the name Oce
Printing Systems GmbH nor the
names<BR>+ of the authors may be used
to endorse or promote products<BR>+
derived from this software without specific prior
written<BR>+ permission.<BR>+<BR>+
THIS SOFTWARE IS PROVIDED “AS IS” AND ANY EXPRESS OR IMPLIED<BR>+
WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE WARRANTIES OF<BR>+
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AND<BR>+ NON-INFRINGEMENT
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ OR CONTRIBUTOR OR COPYRIGHT
HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
OR CONSEQUENTIAL<BR>+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT
OF<BR>+ SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;<BR>+ OR
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF<BR>+ LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+ (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY<BR>+ OF SUCH DAMAGE.
<BR>+<BR>+#############################################################################<BR>+<BR>+<BR>+GENERAL<BR>+=======<BR>+The
RSockets protocol provides socket-based RDMA communication between<BR>+Windows
nodes (like to WSD or ND) as well as between Windows and Linux
nodes.<BR>+<BR>+The RSockets functionality is contained within the librdmacm.dll
which now<BR>+is capable to act as a Winsock base transport
provider.<BR>+<BR>+For now the librdmacm.dll still exports the direct rsocket
calls<BR>+(rsocket, rbind, rrecv etc.) as well. So application developers
can<BR>+alternatively circumvent Winsock and call those functions
directly<BR>+(by including rsocket.h instead of rwinsock.h).<BR>+Aside from a
slight performance gain, this might be useful in case of<BR>+quickly porting a
Linux app to Windows(?).<BR>+But beware of using both access methods
concurrently in the same
application!<BR>+<BR>+<BR>+INSTALLATION<BR>+============<BR>+Installation of
that Winsock provider (i.e. registration of the<BR>+librdmacm.dll in the Windows
Registry) can be done with the rsinstall<BR>+tool (see tools/rsinstall) which
works similar to wsdinstall for the<BR>+WSD Winsock provider.<BR>+For a list of
available options, just call rsinstall.exe without<BR>+parameters. (Note that
rsinstall.exe requires administrative privileges<BR>+to run
properly!)<BR>+<BR>+<BR>+USAGE<BR>+=====<BR>+Usage of the RSocket provider at
application level is quite simple,<BR>+as demonstrated by the rstream tool (see
ulp/librdmacm/examples/rstream)<BR>+which is also a porting from Linux OFED. In
contrast to a 'normal' Winsock<BR>+application there are just two essential
differences:<BR>+<BR>+- The ulp\librdmacm\include\rdma\rwinsock.h header has to
be included<BR>+ instead of winsock2.h. (Nonetheless it's still necessary
to call<BR>+ WSAStartup() and WSACleanup() during initialization and
shutdown of<BR>+ your application, respectivily).<BR>+<BR>+- Instead of
calling socket() for socket creation, a WSASocket() has to<BR>+ be
performed with a WSAPROTOCOL_INFO structure selecting the appropriate<BR>+
Winsock provider. For convenience there is a little helper function<BR>+
rsGetProtocolInfo() implemented in rwinsock.h which provides this
structure<BR>+ based on the provider's GUID (static variable
'rsProviderGuid' which is<BR>+ also contained in
rwinsock.h).<BR>+<BR>+<BR>+RESTRICTIONS<BR>+============<BR>+Generally there are
the same restrictions for socket applications as<BR>+described in the Linux
RSockets man page (e.g. no UDP / SOCK_DGRAM).<BR>+Moreover the following
restrictions apply:<BR>+<BR>+- The MSG_DONTWAIT flag is not supported when
calling WSASocket().<BR>+ Instead to configure a socket for non-blocking
operation,<BR>+ ioctlsocket(FIONBIO) can be used.<BR>+<BR>+- Overlapped
operation is currently not supported, i.e. a WSASocket() with<BR>+ the
WSA_FLAG_OVERLAPPED flag set will be rejected with a WSAEINVAL error.<BR>+<BR>+-
The WSAPoll() function (in Windows Vista and later) is not supported,<BR>+
hence the select() function has to be used instead.<BR>+<BR>+- IPv6 should work,
but has not been tested yet.<BR>Index:
ulp/librdmacm/src/addrinfo.cpp<BR>===================================================================<BR>---
ulp/librdmacm/src/addrinfo.cpp (revision 3419)<BR>+++
ulp/librdmacm/src/addrinfo.cpp (working copy)<BR>@@ -1,11 +1,8
@@<BR> /*<BR> * Copyright (c) 2010 Intel Corporation. All
rights reserved.<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All
rights reserved.<BR> *<BR>- * This software is available to you under a
choice of one of two<BR>- * licenses. You may choose to be licensed under
the terms of the GNU<BR>- * General Public License (GPL) Version 2, available
from the file<BR>- * COPYING in the main directory of this source tree, or
the<BR>- * OpenIB.org BSD license below:<BR>+ * This software is available to
you under the BSD license below:<BR> *<BR> *
Redistribution and use in source and binary forms, with or<BR>
* without modification, are permitted provided that the
following<BR>@@ -20,14 +17,24 @@<BR>
* disclaimer in the documentation
and/or other materials<BR> *
provided with the distribution.<BR> *<BR>- * THE SOFTWARE IS PROVIDED "AS
IS", WITHOUT WARRANTY OF ANY KIND,<BR>- * EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO THE WARRANTIES OF<BR>- * MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND<BR>- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS<BR>- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN<BR>- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN<BR>-
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<BR>- *
SOFTWARE.<BR>+ * - Neither the name Oce Printing
Systems GmbH nor the names<BR>+ * of
the authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR> */<BR> <BR> #if HAVE_CONFIG_H<BR>@@ -47,7 +54,7
@@<BR> {<BR> WSADATA
wsadata;<BR> <BR>- EnterCriticalSection(&lock);<BR>+ fastlock_acquire(&lock);<BR> if
(addr_ref++) {<BR> goto out;<BR> }<BR>@@ -57,16
+64,16
@@<BR> }<BR> <BR> out:<BR>- LeaveCriticalSection(&lock);<BR>+ fastlock_release(&lock);<BR> }<BR> <BR> static
void
ucma_shutdown(void)<BR> {<BR>- EnterCriticalSection(&lock);<BR>+ fastlock_acquire(&lock);<BR> if
(--addr_ref == 0)
{<BR> WSACleanup();<BR> }<BR>- LeaveCriticalSection(&lock);<BR>+ fastlock_release(&lock);<BR> }<BR> <BR> static
void ucma_convert_to_ai(struct addrinfo *ai, struct rdma_addrinfo
*rai)<BR>Index:
ulp/librdmacm/src/cma.cpp<BR>===================================================================<BR>---
ulp/librdmacm/src/cma.cpp (revision 3419)<BR>+++
ulp/librdmacm/src/cma.cpp (working copy)<BR>@@ -1,8 +1,8
@@<BR> /*<BR> * Copyright (c) 2005-2009 Intel Corporation. All
rights reserved.<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All
rights reserved.<BR> *<BR>- * This software is available to you under the
OpenIB.org BSD license<BR>- * below:<BR>+ * This software is available to you
under the BSD license below:<BR> *<BR> *
Redistribution and use in source and binary forms, with or<BR>
* without modification, are permitted provided that the
following<BR>@@ -17,21 +17,31 @@<BR>
* disclaimer in the documentation
and/or other materials<BR> *
provided with the distribution.<BR> *<BR>- * THE SOFTWARE IS PROVIDED "AS
IS", WITHOUT WARRANTY OF ANY KIND,<BR>- * EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO THE WARRANTIES OF<BR>- * MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AWV<BR>- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS<BR>- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN<BR>- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN<BR>-
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<BR>- *
SOFTWARE.<BR>+ * - Neither the name Oce Printing
Systems GmbH nor the names<BR>+ * of
the authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR> */<BR> <BR> #include <windows.h><BR> #include
<winsock2.h><BR>+#include "openib_osd.h"<BR> #include
<stdio.h><BR> #include <iphlpapi.h><BR>-<BR> #include
<rdma/rdma_cma.h><BR> #include
<rdma/rdma_verbs.h><BR> #include <infiniband/verbs.h><BR>@@
-88,6 +98,7
@@<BR> int port_cnt;<BR> uint8_t max_initiator_depth;<BR> uint8_t max_responder_resources;<BR>+ int max_qpsize;<BR> };<BR> <BR> struct
cma_event {<BR>@@ -100,6 +111,59 @@<BR> static int
cma_dev_cnt;<BR> static DWORD ref;<BR> <BR>+void wsa_setlasterror(int
err)<BR>+{<BR>+ int wsa_err = 0;<BR>+ switch (err)
{<BR>+ case
0: break;<BR>+ case
EADDRINUSE: {wsa_err =
WSAEADDRINUSE; break;}<BR>+ case
EADDRNOTAVAIL: {wsa_err =
WSAEADDRNOTAVAIL; break;}<BR>+ case
EAFNOSUPPORT: {wsa_err =
WSAEAFNOSUPPORT; break;}<BR>+ case
EALREADY: {wsa_err =
WSAEALREADY; break;}<BR>+// case
EBADMSG: {wsa_err = ; break;}<BR>+ case
ECANCELED: {wsa_err =
WSAECANCELLED; break;}<BR>+ case
ECONNABORTED: {wsa_err =
WSAECONNABORTED; break;}<BR>+ case
ECONNREFUSED: {wsa_err =
WSAECONNREFUSED; break;}<BR>+ case
ECONNRESET: {wsa_err =
WSAECONNRESET; break;}<BR>+ case
EDESTADDRREQ: {wsa_err =
WSAEDESTADDRREQ; break;}<BR>+ case
EHOSTUNREACH: {wsa_err =
WSAEHOSTUNREACH; break;}<BR>+// case
EIDRM: {wsa_err = ; break;}<BR>+ case
EINPROGRESS: {wsa_err =
WSAEINPROGRESS; break;}<BR>+ case
EISCONN: {wsa_err =
WSAEISCONN; break;}<BR>+ case
ELOOP: {wsa_err =
WSAELOOP; break;}<BR>+ case
EMSGSIZE: {wsa_err =
WSAEMSGSIZE; break;}<BR>+ case
ENETDOWN: {wsa_err =
WSAENETDOWN; break;}<BR>+ case
ENETRESET: {wsa_err =
WSAENETRESET; break;}<BR>+ case
ENETUNREACH: {wsa_err =
WSAENETUNREACH; break;}<BR>+ case
ENOBUFS: {wsa_err =
WSAENOBUFS; break;}<BR>+// case
ENODATA: {wsa_err = ; break;}<BR>+// case
ENOLINK: {wsa_err = ; break;}<BR>+// case
ENOMSG: {wsa_err = ; break;}<BR>+ case
ENOPROTOOPT: {wsa_err =
WSAENOPROTOOPT; break;}<BR>+// case
ENOSR: {wsa_err = ; break;}<BR>+// case
ENOSTR: {wsa_err = ; break;}<BR>+ case
ENOTCONN: {wsa_err =
WSAENOTCONN; break;}<BR>+ case
ENOTRECOVERABLE: {wsa_err =
WSANO_RECOVERY; break;}<BR>+ case
ENOTSOCK: {wsa_err =
WSAENOTSOCK; break;}<BR>+ case
ENOTSUP: {wsa_err = WSAEINVAL; break;}
//???<BR>+ case EOPNOTSUPP: {wsa_err =
WSAEOPNOTSUPP; break;}<BR>+// case
EOTHER: {wsa_err = ; break;}<BR>+// case
EOVERFLOW: {wsa_err = ; break;}<BR>+// case
EOWNERDEAD: {wsa_err = ; break;}<BR>+ case
EPROTO: {wsa_err = WSAENOPROTOOPT; break;}
//???<BR>+ case EPROTONOSUPPORT: {wsa_err =
WSAEPROTONOSUPPORT; break;}<BR>+ case
EPROTOTYPE: {wsa_err =
WSAEPROTOTYPE; break;}<BR>+// case
ETIME: {wsa_err = ; break;}<BR>+ case
ETIMEDOUT: {wsa_err =
WSAETIMEDOUT; break;}<BR>+// case
ETXTBSY: {wsa_err = ; break;}<BR>+ case
ENOMEM: {wsa_err = WSA_NOT_ENOUGH_MEMORY;
break;}<BR>+ case EAGAIN:<BR>+ case
EWOULDBLOCK: {wsa_err =
WSAEWOULDBLOCK; break;}<BR>+ default:
;<BR>+ }<BR>+ WSASetLastError(wsa_err);<BR>+}<BR>+<BR> static int
ucma_acquire(void)<BR> {<BR> struct ibv_device **dev_list =
NULL;<BR>@@ -107,7 +171,7 @@<BR> struct ibv_device_attr
attr;<BR> int i, ret,
dev_cnt;<BR> <BR>- EnterCriticalSection(&lock);<BR>+ fastlock_acquire(&lock);<BR> if
(ref++) {<BR> goto out;<BR> }<BR>@@ -153,13 +217,14
@@<BR> }<BR> <BR> cma_dev->port_cnt =
attr.phys_port_cnt;<BR>+ cma_dev->max_qpsize =
attr.max_qp_wr;<BR> cma_dev->max_initiator_depth = (uint8_t)
attr.max_qp_init_rd_atom;<BR> cma_dev->max_responder_resources
= (uint8_t)
attr.max_qp_rd_atom;<BR> }<BR> ibv_free_device_list(dev_list);<BR> cma_dev_cnt
=
dev_cnt;<BR> out:<BR>- LeaveCriticalSection(&lock);<BR>+ fastlock_release(&lock);<BR> return
0;<BR> <BR> err4:<BR>@@ -174,7 +239,7
@@<BR> ibvw_release_windata(&windata,
IBVW_WINDATA_VERSION);<BR> err1:<BR> ref--;<BR>- LeaveCriticalSection(&lock);<BR>+ fastlock_release(&lock);<BR> return
ret;<BR> }<BR> <BR>@@ -182,7 +247,7 @@<BR> {<BR> int
i;<BR> <BR>- EnterCriticalSection(&lock);<BR>+ fastlock_acquire(&lock);<BR> if
(--ref == 0) {<BR> for (i = 0; i < cma_dev_cnt; i++)
{<BR> ibv_dealloc_pd(cma_dev_array[i].pd);<BR>@@ -192,7
+257,7 @@<BR> cma_dev_cnt =
0;<BR> ibvw_release_windata(&windata,
IBVW_WINDATA_VERSION);<BR> }<BR>- LeaveCriticalSection(&lock);<BR>+ fastlock_release(&lock);<BR> }<BR> <BR> __declspec(dllexport)<BR>@@
-329,9 +394,9 @@<BR> <BR> id_priv = CONTAINING_RECORD(id, struct
cma_id_private,
id);<BR> <BR>- EnterCriticalSection(&lock);<BR>+ fastlock_acquire(&lock);<BR> id_priv->state
=
cma_destroying;<BR>- LeaveCriticalSection(&lock);<BR>+ fastlock_release(&lock);<BR> <BR> if
(id->ps == RDMA_PS_TCP)
{<BR> id->ep.connect->CancelOverlappedRequests();<BR>@@
-502,13 +567,20 @@<BR> return ret;<BR> }<BR> <BR>-static
int ucma_complete(struct cma_id_private *id_priv)<BR>+static int
ucma_complete_priv(struct cma_id_private *id_priv)<BR> {<BR>+ return
ucma_complete(&id_priv->id);<BR>+}<BR>+<BR>+int ucma_complete(struct
rdma_cm_id *id)<BR>+{<BR>+ struct cma_id_private
*id_priv;<BR> int ret;<BR> <BR>- if (!id_priv->sync)
{<BR>+ id_priv = container_of(id, struct cma_id_private,
id);<BR>+<BR>+ if (!id_priv->sync)<BR> return
0;<BR>- }<BR> <BR> if (id_priv->id.event)
{<BR> rdma_ack_cm_event(id_priv->id.event);<BR>@@ -516,11
+588,18 @@<BR> }<BR> <BR> ret =
rdma_get_cm_event(id_priv->id.channel,
&id_priv->id.event);<BR>- if (ret) {<BR>+ if
(ret)<BR> return ret;<BR>+<BR>+ if
(id_priv->id.event->status) {<BR>+ if
(id_priv->id.event->event ==
RDMA_CM_EVENT_REJECTED)<BR>+ ret =
ERR(ECONNREFUSED);<BR>+ else if (id_priv->id.event->status <
0)<BR>+ ret =
ERR(-id_priv->id.event->status);<BR>+ else<BR>+ ret
= ERR(-id_priv->id.event->status);<BR> }<BR>-<BR>- return
id_priv->id.event->status;<BR>+ return
ret;<BR> }<BR> <BR> __declspec(dllexport)<BR>@@ -561,12 +640,17
@@<BR> }<BR> }<BR> <BR>- RtlCopyMemory(&id->route.addr.dst_addr,
dst_addr, ucma_addrlen(dst_addr));<BR>+ if (((struct sockaddr_in
*)&id->route.addr.dst_addr)->sin_port ==
0)<BR>+ {<BR>+ // port = 0 => Assume that entire dst_addr
hasn't been set
yet<BR>+ RtlCopyMemory(&id->route.addr.dst_addr, dst_addr,
ucma_addrlen(dst_addr));<BR>+ }<BR>+<BR> id_priv->state =
cma_addr_resolve;<BR> <BR> id_priv->refcnt++;<BR> CompEntryPost(&id->comp_entry);<BR>- return
ucma_complete(id_priv);<BR>+ return
ucma_complete_priv(id_priv);<BR> }<BR> <BR> __declspec(dllexport)<BR>@@
-594,7 +678,7
@@<BR> <BR> id_priv->refcnt++;<BR> CompEntryPost(&id->comp_entry);<BR>- return
ucma_complete(id_priv);<BR>+ return
ucma_complete_priv(id_priv);<BR> }<BR> <BR> static int
ucma_modify_qp_init(struct cma_id_private *id_priv, struct ibv_qp *qp)<BR>@@
-648,10 +732,10 @@<BR> if
(id->recv_cq_channel)<BR> ibv_destroy_comp_channel(id->recv_cq_channel);<BR> <BR>- if
(id->send_cq)<BR>+ if (id->send_cq && id->send_cq !=
id->recv_cq)<BR> ibv_destroy_cq(id->send_cq);<BR> <BR>- if
(id->send_cq_channel)<BR>+ if (id->send_cq_channel &&
id->send_cq_channel !=
id->recv_cq_channel)<BR> ibv_destroy_comp_channel(id->send_cq_channel);<BR> }<BR> <BR>@@
-832,7 +916,7 @@<BR> return
ibvw_wv_errno(hr);<BR> }<BR> <BR>- return
ucma_complete(id_priv);<BR>+ return
ucma_complete_priv(id_priv);<BR> }<BR> <BR> static int
ucma_get_request(struct cma_id_private *listen, int index)<BR>@@ -841,7 +925,7
@@<BR> HRESULT hr;<BR> int
ret;<BR> <BR>- EnterCriticalSection(&lock);<BR>+ fastlock_acquire(&lock);<BR> if
(listen->state != cma_listening) {<BR> ret =
ibvw_wv_errno(WV_INVALID_PARAMETER);<BR> goto err1;<BR>@@
-873,14 +957,14
@@<BR> id_priv->refcnt--;<BR> goto
err2;<BR> }<BR>- LeaveCriticalSection(&lock);<BR>+ fastlock_release(&lock);<BR> <BR> return
0;<BR> <BR> err2:<BR> InterlockedDecrement(&listen->refcnt);<BR> err1:<BR>- LeaveCriticalSection(&lock);<BR>+ fastlock_release(&lock);<BR> if
(id_priv != NULL)
{<BR> rdma_destroy_id(&id_priv->id);<BR> }<BR>@@
-1011,7 +1095,7 @@<BR> return
ibvw_wv_errno(hr);<BR> }<BR> <BR>- return
ucma_complete(id_priv);<BR>+ return
ucma_complete_priv(id_priv);<BR> }<BR> <BR> __declspec(dllexport)<BR>@@
-1053,7 +1137,7 @@<BR> return
ibvw_wv_errno(hr);<BR> }<BR> <BR>- return
ucma_complete(id_priv);<BR>+ return
ucma_complete_priv(id_priv);<BR> }<BR> <BR> __declspec(dllexport)<BR>@@
-1179,7 +1263,7 @@<BR> <BR> id_priv =
event->id_priv;<BR> <BR>- EnterCriticalSection(&lock);<BR>+ fastlock_acquire(&lock);<BR> switch
(id_priv->state) {<BR> case
cma_get_request:<BR> listen = (struct cma_id_private *)
id_priv->id.context;<BR>@@ -1190,7 +1274,7
@@<BR> }<BR> <BR> listen->req_list[id_priv->index]
=
NULL;<BR>- LeaveCriticalSection(&lock);<BR>+ fastlock_release(&lock);<BR> return
ucma_process_conn_req(event);<BR> case
cma_addr_resolve:<BR> event->event.event =
RDMA_CM_EVENT_ADDR_RESOLVED;<BR>@@ -1216,7 +1300,7
@@<BR> InterlockedDecrement(&id_priv->refcnt);<BR> ret
=
ECANCELED;<BR> }<BR>- LeaveCriticalSection(&lock);<BR>+ fastlock_release(&lock);<BR> <BR> return
ret;<BR> }<BR>@@ -1240,6 +1324,8 @@<BR> <BR> ret =
CompChannelPoll(&channel->channel, &entry);<BR> if
(ret) {<BR>+ if (ret ==
WAIT_TIMEOUT)<BR>+ ret =
ERR(EWOULDBLOCK);<BR> delete
evt;<BR> return ret;<BR> }<BR>@@ -1473,3
+1559,11 @@<BR> }<BR> return -1;<BR> }<BR>+<BR>+int
ucma_max_qpsize(struct rdma_cm_id *id)<BR>+{<BR>+ struct cma_id_private
*id_priv;<BR>+<BR>+ id_priv = container_of(id, struct cma_id_private,
id);<BR>+ return id_priv->cma_dev->max_qpsize;<BR>+}<BR>Index:
ulp/librdmacm/src/cma.h<BR>===================================================================<BR>---
ulp/librdmacm/src/cma.h (revision 3419)<BR>+++
ulp/librdmacm/src/cma.h (working copy)<BR>@@ -1,9 +1,9
@@<BR> /*<BR> * Copyright (c) 2004, 2005 Topspin
Communications. All rights reserved.<BR> * Copyright (c) 2008-2009
Intel Corp. All rights reserved.<BR>+ * Copyright (c) 2012 Oce Printing
Systems GmbH. All rights reserved.<BR> *<BR>- * This software is
available to you under the OpenIB.org BSD license<BR>- * below:<BR>+ * This
software is available to you under the BSD license below:<BR> *<BR>
* Redistribution and use in source and binary forms,
with or<BR> * without modification, are permitted
provided that the following<BR>@@ -18,24 +18,60 @@<BR>
* disclaimer in the documentation
and/or other materials<BR> *
provided with the distribution.<BR> *<BR>- * THE SOFTWARE IS PROVIDED "AS
IS", WITHOUT WARRANTY OF ANY KIND,<BR>- * EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO THE WARRANTIES OF<BR>- * MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AWV<BR>- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS<BR>- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN<BR>- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN<BR>-
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<BR>- *
SOFTWARE.<BR>+ * - Neither the name Oce Printing
Systems GmbH nor the names<BR>+ * of
the authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR> */<BR> <BR> #ifndef CMA_H<BR> #define
CMA_H<BR> <BR>-extern CRITICAL_SECTION lock;<BR>-extern HANDLE
heap;<BR>+#include <complib/cl_spinlock.h><BR>+#include
<rdma/rdma_verbs.h><BR> <BR>+/*<BR>+ * Fast synchronization for low
contention locking.<BR>+ */<BR>+#define
fastlock_t cl_spinlock_t<BR>+#define
fastlock_init(lock) cl_spinlock_init(lock)<BR>+#define
fastlock_destroy(lock) cl_spinlock_destroy(lock)<BR>+#define
fastlock_acquire(lock) cl_spinlock_acquire(lock)<BR>+#define
fastlock_release(lock) cl_spinlock_release(lock)<BR>+<BR>+extern fastlock_t
lock;<BR>+extern HANDLE heap;<BR>+<BR>+#define
TRACE(fmt, ...) Trace(__FUNCTION__": " fmt "\n", __VA_ARGS__)<BR>+<BR>+void
Trace(const char* fmt, ...);<BR> void ucma_cleanup();<BR>+int
ucma_max_qpsize(struct rdma_cm_id *id);<BR>+int ucma_complete(struct rdma_cm_id
*id);<BR>+void wsa_setlasterror(int err);<BR> <BR>+static __inline int
ERR(int err)<BR>+{<BR>+ int ret =
rdma_seterrno(err);<BR>+ if
(ret)<BR>+ wsa_setlasterror(err);<BR>+ return ret;<BR>+}<BR>+
<BR> __inline void* __cdecl operator new(size_t
size)<BR> {<BR> return HeapAlloc(heap, 0, size);<BR>@@ -46,4
+82,13 @@<BR> HeapFree(heap, 0,
pObj);<BR> }<BR> <BR>+#ifndef SYSCONFDIR<BR>+#define SYSCONFDIR
"/etc"<BR>+#endif<BR>+#ifndef RDMADIR<BR>+#define RDMADIR
"rdma"<BR>+#endif<BR>+#define RDMA_CONF_DIR SYSCONFDIR "/"
RDMADIR<BR>+#define RS_CONF_DIR RDMA_CONF_DIR "/rsocket"<BR>+<BR> #endif /*
CMA_H */<BR>Index:
ulp/librdmacm/src/cma_exports.src<BR>===================================================================<BR>---
ulp/librdmacm/src/cma_exports.src (revision 3419)<BR>+++
ulp/librdmacm/src/cma_exports.src (working copy)<BR>@@ -30,4 +30,30
@@<BR> rdma_freeaddrinfo<BR> rdma_get_request<BR> rdmaw_wsa_errno<BR>+rsocket<BR>+rbind<BR>+rlisten<BR>+raccept<BR>+rconnect<BR>+rshutdown<BR>+rclose<BR>+rrecv<BR>+rrecvfrom<BR>+rrecvmsg<BR>+rsend<BR>+rsendto<BR>+rsendmsg<BR>+rread<BR>+rreadv<BR>+rwrite<BR>+rwritev<BR>+rgetpeername<BR>+rgetsockname<BR>+rsetsockopt<BR>+rgetsockopt<BR>+rfcntl<BR>+rioctlsocket<BR>+rselect<BR>+rpoll<BR>+WSPStartup<BR> #endif<BR>Index:
ulp/librdmacm/src/cma_main.cpp<BR>===================================================================<BR>---
ulp/librdmacm/src/cma_main.cpp (revision 3419)<BR>+++
ulp/librdmacm/src/cma_main.cpp (working copy)<BR>@@ -1,8 +1,8
@@<BR> /*<BR>- * Copyright (c) 2008-2009 Intel Corporation. All rights
reserved.<BR>+ * Copyright (c) 2008-2009 Intel Corporation. All rights
reserved.<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All rights
reserved.<BR> *<BR>- * This software is available to you under the
OpenIB.org BSD license<BR>- * below:<BR>+ * This software is available to you
under the BSD license below:<BR> *<BR> *
Redistribution and use in source and binary forms, with or<BR>
* without modification, are permitted provided that the
following<BR>@@ -17,22 +17,1173 @@<BR>
* disclaimer in the documentation
and/or other materials<BR> *
provided with the distribution.<BR> *<BR>- * THE SOFTWARE IS PROVIDED "AS
IS", WITHOUT WARRANTY OF ANY KIND,<BR>- * EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO THE WARRANTIES OF<BR>- * MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AWV<BR>- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS<BR>- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN<BR>- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN<BR>-
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<BR>- *
SOFTWARE.<BR>+ * - Neither the name Oce Printing
Systems GmbH nor the names<BR>+ * of
the authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR> */<BR>-<BR>-#include <windows.h><BR>+ <BR>+#include
"openib_osd.h" <BR> #include "cma.h"<BR>+#include
"rdma/rwinsock.h"<BR>+#include "rdma/rsocket.h"<BR>+#include
<complib/cl_debug.h><BR>+#include "../../../etc/user/gtod.c" //
getimeofday()<BR> <BR>-CRITICAL_SECTION lock;<BR>-HANDLE heap;<BR>+#include
<strsafe.h><BR>+#include <stdio.h><BR>+#include
<stdlib.h><BR> <BR>+#define MAX_TRACE_INDEX 4096 // Has to be a power
of 2<BR>+<BR>+typedef struct {<BR>+ FILETIME
fTime;<BR>+ char* pszFormat;<BR>+ ULONG64
qwArgs[4];<BR>+} RS_TRACE, *LPRS_TRACE;<BR>+<BR>+/*<BR>+ * Globals used across
files<BR>+ */<BR>+CRITICAL_SECTION
gCriticalSection; // Critical section to protect
startup/cleanup<BR>+WSPUPCALLTABLE
gMainUpCallTable; // Winsock upcall
table<BR>+WSAPROTOCOL_INFOW gProtocolInfo;<BR>+BOOL gDetached
= FALSE; // Indicates if process is detaching from
DLL<BR>+fastlock_t lock;<BR>+fastlock_t mut;<BR>+HANDLE heap;<BR>+<BR>+/*<BR>+
* Globals local to this file<BR>+ */<BR>+static const WCHAR
*Description = L"OpenFabrics RSockets for
InfiniBand";<BR>+static
int gStartupCount =
0; // Global startup count (for every WSPStartup
call)<BR>+static uint32_t
iTrace = 0;<BR>+static
RS_TRACE rsTrace[MAX_TRACE_INDEX] = {0};<BR>+static
fastlock_t TraceLock;<BR>+<BR>+/*<BR>+ * Create trace
entries at runtime<BR>+ */<BR>+void Trace (const char* fmt,
...)<BR>+{<BR>+ struct
timeval time;<BR>+ va_list argptr;<BR>+ int i;<BR>+<BR>+ fastlock_acquire(&TraceLock);<BR>+<BR>+ GetSystemTimeAsFileTime(&rsTrace[iTrace].fTime);<BR>+ rsTrace[iTrace].pszFormat
= (char *)fmt;<BR>+<BR>+ va_start(argptr, fmt);<BR>+ for (i = 0; i
< sizeof(rsTrace[0].qwArgs) / sizeof(rsTrace[0].qwArgs[0]);
i++)<BR>+ rsTrace[iTrace].qwArgs[i] = va_arg(argptr,
uint64_t);<BR>+ va_end(argptr);<BR>+<BR>+ iTrace = (iTrace + 1) &
(MAX_TRACE_INDEX-1);<BR>+<BR>+ fastlock_release(&TraceLock);<BR>+ <BR>+ return;<BR>+}<BR>+<BR>+/**<BR>+
* \brief Get current RSockets trace information for
the calling process<BR>+ * (by calling
librdmacm.dll directly).<BR>+ *<BR>+ * \param lppTraceBuffer Pointer to a
buffer with an array of RS_TRACE information entries<BR>+
* to be allocated and returned. The
caller is responsible for<BR>+
* deallocating that buffer via free()
when it is no longer needed.<BR>+ *<BR>+ *
\return The number of RS_TRACE entries contained in
the trace buffer<BR>+ * returned by
*lppTraceBuffer.<BR>+ */<BR>+static uint32_t rsGetTrace ( __out LPRS_TRACE_OUT
*lppTraceBuffer )<BR>+{<BR>+ uint32_t i, e, count = 0;<BR>+ struct
timeval tvTime;<BR>+<BR>+ if (NULL == lppTraceBuffer
||<BR>+ NULL == (*lppTraceBuffer =
(LPRS_TRACE_OUT)malloc(MAX_TRACE_INDEX *
sizeof(**lppTraceBuffer)))<BR>+ )<BR>+ return
0;<BR>+<BR>+ memset( *lppTraceBuffer, 0, MAX_TRACE_INDEX *
sizeof(**lppTraceBuffer)
);<BR>+ <BR>+ fastlock_acquire(&TraceLock);<BR>+ for
(<BR>+ i = 0, e = iTrace;<BR>+ i <
MAX_TRACE_INDEX;<BR>+ i++, e = (e+1) & (MAX_TRACE_INDEX -
1)<BR>+ )<BR>+ {<BR>+ if
(rsTrace[e].pszFormat)<BR>+ {<BR>+ FileTimeToTimeval(&rsTrace[e].fTime,
&tvTime);<BR>+ sprintf_s(<BR>+
(*lppTraceBuffer)[i].szLine,<BR>+ sizeof((*lppTraceBuffer)[0].szLine),<BR>+ "%010d:%010d
%s",<BR>+ tvTime.tv_sec,
tvTime.tv_usec,<BR>+ rsTrace[e].pszFormat,<BR>+ rsTrace[e].qwArgs[0],
rsTrace[e].qwArgs[1], rsTrace[e].qwArgs[2],
rsTrace[e].qwArgs[3]<BR>+ );<BR>+ cl_dbg_out("%s",
(*lppTraceBuffer)[i].szLine);<BR>+ count++;<BR>+ }<BR>+ }<BR>+ fastlock_release(&TraceLock);<BR>+ <BR>+ return
count;<BR>+}<BR>+<BR>+/*<BR>+ * SPI Function Implementation<BR>+ */<BR>+<BR>+/*
<BR>+ * Function: WSPCleanup<BR>+ *<BR>+ * Description:<BR>+ *
Decrement the entry count. If equal to zero then we can prepare to have us<BR>+
* unloaded so all resources should be freed<BR>+ */<BR>+int
WSPAPI <BR>+WSPCleanup(<BR>+ LPINT
lpErrno <BR>+
)<BR>+{<BR>+ int rc =
SOCKET_ERROR;<BR>+<BR>+ if ( gDetached )
{<BR>+ rc =
NO_ERROR;<BR>+ goto
cleanup;<BR>+ }<BR>+<BR>+
//<BR>+ // Grab the DLL global critical
section<BR>+ //<BR>+ EnterCriticalSection(
&gCriticalSection );<BR>+<BR>+ // Verify WSPStartup has
been called<BR>+ if ( 0 == gStartupCount )
{<BR>+ *lpErrno =
WSANOTINITIALISED;<BR>+ goto
cleanup;<BR>+ }<BR>+<BR>+ // Decrement the
global entry count<BR>+
gStartupCount--;<BR>+<BR>+TRACE("StartupCount decremented to %d",
gStartupCount);<BR>+<BR>+ if ( 0 == gStartupCount )
{<BR>+ // Free LSP structures if still
present as well as call
WSPCleanup<BR>+ //
for all providers this LSP loaded<BR>+
}<BR>+ rc = NO_ERROR;<BR>+<BR>+cleanup:<BR>+
LeaveCriticalSection( &gCriticalSection );<BR>+<BR>+
return rc;<BR>+}<BR>+<BR>+/*<BR>+ * Function: WSPSocket<BR>+ *<BR>+ *
Description:<BR>+ * This routine creates a socket. For an IFS
LSP the lower provider's socket<BR>+ * handle is returned to
the uppler layer. When a socket is created, a socket<BR>+ *
context structure is created for the socket returned from the lower
provider.<BR>+ * This context is used if the socket is later
connected to a proxied address.<BR>+ */<BR>+SOCKET WSPAPI
<BR>+WSPSocket(<BR>+
int
af,<BR>+
int
type,<BR>+
int
protocol,<BR>+ LPWSAPROTOCOL_INFOW
lpProtocolInfo,<BR>+
GROUP
g,<BR>+
DWORD
dwFlags,<BR>+
LPINT
lpErrno<BR>+
)<BR>+{<BR>+ WSAPROTOCOL_INFOW InfoCopy =
{0};<BR>+ int rs =
(int)INVALID_SOCKET;<BR>+
SOCKET
winSocket =
INVALID_SOCKET,<BR>+
sret = INVALID_SOCKET;<BR>+
int
rc;<BR>+<BR>+TRACE("af=%d type=%d protocol=%d flags=0x%X", af,
type, protocol, dwFlags);<BR>+<BR>+ if (af != AF_INET)
{<BR>+ *lpErrno = WSAEAFNOSUPPORT;<BR>+ return
INVALID_SOCKET;<BR>+ }<BR>+<BR>+ if (type != SOCK_STREAM)
{<BR>+ *lpErrno = WSAEPROTOTYPE;<BR>+ return
INVALID_SOCKET;<BR>+ }<BR>+<BR>+ if (protocol != IPPROTO_TCP)
{<BR>+ *lpErrno = WSAEPROTONOSUPPORT;<BR>+ return
INVALID_SOCKET;<BR>+ }<BR>+<BR>+ if (dwFlags != 0)
{<BR>+ *lpErrno = WSAEINVAL;<BR>+ return
INVALID_SOCKET;<BR>+ }<BR>+<BR>+
//<BR>+ // Create the socket from the lower
layer<BR>+ //<BR>+ if ( INVALID_SOCKET == (rs = rsocket(
af, type, protocol)) ) {<BR>+ *lpErrno =
WSAGetLastError();<BR>+ goto
cleanup;<BR>+ }<BR>+<BR>+ winSocket =
gMainUpCallTable.lpWPUCreateSocketHandle(<BR>+ gProtocolInfo.dwCatalogEntryId,<BR>+ rs, //
__in DWORD_PTR
dwContext<BR>+ lpErrno<BR>+ );<BR>+ if
(INVALID_SOCKET == winSocket)<BR>+
goto cleanup;<BR>+<BR>+TRACE(" New socket handle = %d",
winSocket);<BR>+<BR>+ return
winSocket;<BR>+<BR>+cleanup:<BR>+<BR>+ // If an error occured
close the socket if it was already created<BR>+ if
(INVALID_SOCKET != rs)<BR>+
rclose((int)rs);<BR>+ <BR>+ *lpErrno =
WSAGetLastError();<BR>+<BR>+ return
INVALID_SOCKET;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPBind(<BR>+ __in SOCKET
s,<BR>+ __in const struct sockaddr*
name,<BR>+ __in int namelen,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ struct sockaddr_in * name_in = (struct
sockaddr_in *)name;<BR>+ struct sockaddr_in6* name_in6 = (struct
sockaddr_in6*)name;<BR>+<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ ret = rbind((int)rs, name, namelen);<BR>+ if
(SOCKET_ERROR == ret)<BR>+ *lpErrno =
WSAGetLastError();<BR>+<BR>+TRACE("Socket = %d:", s);<BR>+ if (AF_INET ==
name->sa_family) {<BR>+ TRACE(" Addr =
%d.%d.%d.%d",<BR>+ name_in->sin_addr.S_un.S_un_b.s_b1,<BR>+ name_in->sin_addr.S_un.S_un_b.s_b2,<BR>+ name_in->sin_addr.S_un.S_un_b.s_b3,<BR>+ name_in->sin_addr.S_un.S_un_b.s_b4<BR>+ );<BR>+ TRACE("
Port = %d, Returning %d", name_in->sin_port, ret);<BR>+ }
else {<BR>+ TRACE(" Addr =
%d:%d:%d:%d:%d:%d:%d:%d",<BR>+ name_in6->sin6_addr.u.Word[0],<BR>+ name_in6->sin6_addr.u.Word[1],<BR>+ name_in6->sin6_addr.u.Word[2],<BR>+ name_in6->sin6_addr.u.Word[3],<BR>+ name_in6->sin6_addr.u.Word[4],<BR>+ name_in6->sin6_addr.u.Word[5],<BR>+ name_in6->sin6_addr.u.Word[6],<BR>+ name_in6->sin6_addr.u.Word[7]<BR>+ );<BR>+ TRACE("
Port = %d , Returning %d", name_in6->sin6_port,
ret);<BR>+ }<BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPListen(<BR>+ __in SOCKET
s,<BR>+ __in int backlog,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+ <BR>+ ret = rlisten((int)rs, backlog);<BR>+ if
(SOCKET_ERROR == ret)<BR>+ *lpErrno =
WSAGetLastError();<BR>+<BR>+TRACE("Socket = %d: Backlog=%d, Returning %d", s,
backlog, ret);<BR>+<BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPConnect(<BR>+ __in SOCKET
s,<BR>+ __in const struct sockaddr*
name,<BR>+ __in int namelen,<BR>+ __in
LPWSABUF lpCallerData,<BR>+ __out LPWSABUF
lpCalleeData,<BR>+ __in LPQOS
lpSQOS,<BR>+ __in LPQOS lpGQOS,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ struct sockaddr_in * name_in = (struct
sockaddr_in *)name;<BR>+ struct sockaddr_in6* name_in6 = (struct
sockaddr_in6*)name;<BR>+<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+ <BR>+ ret = rconnect((int)rs, name, namelen);<BR>+ if
(SOCKET_ERROR == ret)<BR>+ *lpErrno =
WSAGetLastError();<BR>+<BR>+ if
(lpCalleeData)<BR>+ lpCalleeData->len = 0;<BR>+<BR>+TRACE("Socket
= %d:", s);<BR>+ if (AF_INET == name->sa_family)
{<BR>+ TRACE(" Addr =
%d.%d.%d.%d",<BR>+ name_in->sin_addr.S_un.S_un_b.s_b1,<BR>+ name_in->sin_addr.S_un.S_un_b.s_b2,<BR>+ name_in->sin_addr.S_un.S_un_b.s_b3,<BR>+ name_in->sin_addr.S_un.S_un_b.s_b4<BR>+ );<BR>+ TRACE("
Port = %d, Returning %d", name_in->sin_port, ret);<BR>+ }
else {<BR>+ TRACE(" Addr =
%d:%d:%d:%d:%d:%d:%d:%d",<BR>+ name_in6->sin6_addr.u.Word[0],<BR>+ name_in6->sin6_addr.u.Word[1],<BR>+ name_in6->sin6_addr.u.Word[2],<BR>+ name_in6->sin6_addr.u.Word[3],<BR>+ name_in6->sin6_addr.u.Word[4],<BR>+ name_in6->sin6_addr.u.Word[5],<BR>+ name_in6->sin6_addr.u.Word[6],<BR>+ name_in6->sin6_addr.u.Word[7]<BR>+ );<BR>+ TRACE("
Port = %d , Returning %d", name_in6->sin6_port,
ret);<BR>+ }<BR>+<BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPShutdown(<BR>+ __in SOCKET
s,<BR>+ __in int how,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+ <BR>+ ret = rshutdown((int)rs, how);<BR>+ if
(SOCKET_ERROR == ret)<BR>+ *lpErrno =
WSAGetLastError();<BR>+<BR>+TRACE("Socket = %d: how=%d, Returning %d", s, how,
ret);<BR>+<BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPCloseSocket(<BR>+ __in SOCKET
s,<BR>+ __out LPINT lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+ <BR>+ ret = rclose((int)rs);<BR>+ if (SOCKET_ERROR ==
ret)<BR>+ *lpErrno = WSAGetLastError();<BR>+ <BR>+ ret =
gMainUpCallTable.lpWPUCloseSocketHandle(s, lpErrno);<BR>+<BR>+TRACE("Socket
handle %s closed", s);<BR>+<BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPRecv(<BR>+ __in
SOCKET s,<BR>+ __inout
LPWSABUF lpBuffers,<BR>+ __in
DWORD dwBufferCount,<BR>+ __out
LPDWORD lpNumberOfBytesRecvd,<BR>+ __inout
LPDWORD lpFlags,<BR>+ __in
LPWSAOVERLAPPED lpOverlapped,<BR>+ __in
LPWSAOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine,<BR>+ __in
LPWSATHREADID lpThreadId,<BR>+ __out
LPINT lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR
rs = INVALID_SOCKET;<BR>+ DWORD
i;<BR>+ DWORD dwNumberOfBytesRecvd =
0;<BR>+ int len =
0;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ for (i = 0; i < dwBufferCount; i++) {<BR>+ len
= (int)rrecv((int)rs, lpBuffers[i].buf, lpBuffers[i].len,
*lpFlags);<BR>+ switch (len) {<BR>+ case
0:<BR>+ goto out;<BR>+ case
-1:<BR>+ *lpErrno =
WSAGetLastError();<BR>+ ret =
SOCKET_ERROR;<BR>+ goto
out;<BR>+ default:<BR>+ dwNumberOfBytesRecvd +=
len;<BR>+ }<BR>+ }<BR>+<BR>+out:<BR>+ *lpNumberOfBytesRecvd
= dwNumberOfBytesRecvd;<BR>+ <BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPSend(<BR>+ __in
SOCKET s,<BR>+ __in
LPWSABUF lpBuffers,<BR>+ __in
DWORD dwBufferCount,<BR>+ __out
LPDWORD lpNumberOfBytesSent,<BR>+ __in
DWORD dwFlags,<BR>+ __in
LPWSAOVERLAPPED lpOverlapped,<BR>+ __in
LPWSAOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine,<BR>+ __in
LPWSATHREADID lpThreadId,<BR>+ __out
LPINT lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR
rs = INVALID_SOCKET;<BR>+ DWORD
i;<BR>+ DWORD dwNumberOfBytesSent =
0;<BR>+ int
len;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ for (i = 0; i < dwBufferCount; i++) {<BR>+ len
= (int)rsend((int)rs, lpBuffers[i].buf, lpBuffers[i].len,
dwFlags);<BR>+ if (-1 == len) {<BR>+ *lpErrno =
WSAGetLastError();<BR>+ ret =
SOCKET_ERROR;<BR>+ break;<BR>+ }
else<BR>+ dwNumberOfBytesSent +=
len;<BR>+ }<BR>+ *lpNumberOfBytesSent =
dwNumberOfBytesSent;<BR>+ <BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPGetSockOpt(<BR>+ __in SOCKET
s,<BR>+ __in int
level,<BR>+ __in int
optname,<BR>+ __out char* optval,<BR>+ __inout
LPINT optlen,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+ <BR>+ ret = rgetsockopt((int)rs, level, optname, optval,
optlen);<BR>+ if (SOCKET_ERROR == ret)<BR>+ *lpErrno =
WSAGetLastError();<BR>+<BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPSetSockOpt(<BR>+ __in SOCKET
s,<BR>+ __in int
level,<BR>+ __in int
optname,<BR>+ __in const char*
optval,<BR>+ __in int
optlen,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ ret = rsetsockopt((int)rs, level, optname, optval,
optlen);<BR>+ if (SOCKET_ERROR == ret)<BR>+ *lpErrno =
WSAGetLastError();<BR>+<BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPSelect(<BR>+ __in int
nfds,<BR>+ __inout fd_set* readfds,<BR>+ __inout fd_set*
writefds,<BR>+ __inout fd_set*
exceptfds,<BR>+ __in const struct timeval*
timeout,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ u_int i;<BR>+ int
ret;<BR>+ fd_set rreadfds, rwritefds,
rexceptfds;<BR>+ <BR>+ FD_ZERO(&rreadfds);<BR>+ FD_ZERO(&rwritefds);<BR>+ FD_ZERO(&rexceptfds);<BR>+ <BR>+ nfds
= 1;<BR>+<BR>+ for (i = 0; readfds && i < readfds->fd_count;
i++) {<BR>+ if (readfds->fd_array[i]) {<BR>+ ret
=
gMainUpCallTable.lpWPUQuerySocketHandleContext(<BR>+
readfds->fd_array[i],<BR>+ &rreadfds.fd_array[i],<BR>+ lpErrno<BR>+ );<BR>+ if
(SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+ <BR>+ if (rreadfds.fd_array[i] >
nfds)<BR>+ nfds =
(int)rreadfds.fd_array[i];<BR>+<BR>+ rreadfds.fd_count++;<BR>+ }<BR>+ }<BR>+ for
(i = 0; writefds && i < writefds->fd_count; i++)
{<BR>+ if (writefds->fd_array[i]) {<BR>+ ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(<BR>+
writefds->fd_array[i],<BR>+ &rwritefds.fd_array[i],<BR>+ lpErrno<BR>+ );<BR>+ if
(SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ if (rwritefds.fd_array[i] >
nfds)<BR>+ nfds =
(int)rwritefds.fd_array[i];<BR>+<BR>+ rwritefds.fd_count++;<BR>+ }<BR>+ }<BR>+ for
(i = 0; exceptfds && i < exceptfds->fd_count; i++)
{<BR>+ if (exceptfds->fd_array[i]) {<BR>+ ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(<BR>+
exceptfds->fd_array[i],<BR>+ &rexceptfds.fd_array[i],<BR>+ lpErrno<BR>+ );<BR>+ if
(SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ if (rexceptfds.fd_array[i] >
nfds)<BR>+ nfds =
(int)rexceptfds.fd_array[i];<BR>+<BR>+ rexceptfds.fd_count++;<BR>+ }<BR>+ }<BR>+ <BR>+ ret
= rselect(<BR>+ nfds + 1, // Max. valid rsocket descriptor +
1<BR>+ readfds ? &rreadfds :
NULL,<BR>+ writefds ? &rwritefds :
NULL,<BR>+ exceptfds ? &rexceptfds :
NULL,<BR>+ (struct
timeval*)timeout<BR>+ );<BR>+ if (SOCKET_ERROR == ret)
{<BR>+ *lpErrno = WSAGetLastError();<BR>+ return
ret;<BR>+ }<BR>+ nfds = ret;<BR>+ <BR>+ for (i = 0; ret
&& readfds && i < rreadfds.fd_count; i++)
{<BR>+ if (rreadfds.fd_array[i] && readfds->fd_array[i])
{<BR>+ ret--;<BR>+ } else
{<BR>+ readfds->fd_array[i] =
0;<BR>+ readfds->fd_count--;<BR>+ }<BR>+ }<BR>+ for
(i = 0; ret && writefds && i < rwritefds.fd_count; i++)
{<BR>+ if (rwritefds.fd_array[i] && writefds->fd_array[i])
{<BR>+ ret--;<BR>+ } else
{<BR>+ writefds->fd_array[i] =
0;<BR>+ writefds->fd_count--;<BR>+ }<BR>+ }<BR>+ for
(i = 0; ret && exceptfds && i < rexceptfds.fd_count; i++)
{<BR>+ if (rexceptfds.fd_array[i] &&
exceptfds->fd_array[i]) {<BR>+ ret--;<BR>+ } else
{<BR>+ exceptfds->fd_array[i] =
0;<BR>+ exceptfds->fd_count--;<BR>+ }<BR>+ }<BR>+ <BR>+ return
(nfds - ret);<BR>+}<BR>+<BR>+int WSPAPI<BR>+WSPIoctl(<BR>+ __in
SOCKET s,<BR>+ __in DWORD dwIoControlCode,<BR>+ __in
LPVOID lpvInBuffer,<BR>+ __in DWORD cbInBuffer,<BR>+ __out
LPVOID lpvOutBuffer,<BR>+ __in DWORD cbOutBuffer,<BR>+ __out
LPDWORD lpcbBytesReturned,<BR>+ __in LPWSAOVERLAPPED
lpOverlapped,<BR>+ __in LPWSAOVERLAPPED_COMPLETION_ROUTINE
lpCompletionRoutine,<BR>+ __in LPWSATHREADID
lpThreadId,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ int ret =
0;<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ DWORD
dwCount;<BR>+ LPVOID lpResultBuffer =
NULL;<BR>+ <BR>+ *lpcbBytesReturned = 0;<BR>+ <BR>+ ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ switch (dwIoControlCode) {<BR>+ case
SIO_RS_GET_STATUS:<BR>+ if (lpvOutBuffer)
{<BR>+ dwCount = rsGetStatus((LPRS_STATUS
*)&lpResultBuffer);<BR>+ if (lpResultBuffer)
{<BR>+ if (cbOutBuffer >= dwCount * sizeof(RS_STATUS))
{<BR>+ *lpcbBytesReturned = dwCount *
sizeof(RS_STATUS);<BR>+ } else
{<BR>+ ret =
SOCKET_ERROR;<BR>+ *lpErrno =
WSA_IO_INCOMPLETE;<BR>+ *lpcbBytesReturned =
cbOutBuffer; // Copy as much as possible
anyway<BR>+ }<BR>+ }<BR>+ }<BR>+ break;<BR>+ case
SIO_RS_GET_TRACE:<BR>+ if (lpvOutBuffer)
{<BR>+ dwCount = rsGetTrace((LPRS_TRACE_OUT
*)&lpResultBuffer);<BR>+ if (lpResultBuffer)
{<BR>+ if (cbOutBuffer >= dwCount *
sizeof(RS_TRACE_OUT)) {<BR>+ *lpcbBytesReturned =
dwCount * sizeof(RS_TRACE_OUT);<BR>+ } else
{<BR>+ ret =
SOCKET_ERROR;<BR>+ *lpErrno =
WSA_IO_INCOMPLETE;<BR>+ *lpcbBytesReturned =
cbOutBuffer; // Copy as much as possible
anyway<BR>+ }<BR>+ }<BR>+ }<BR>+ break;<BR>+ case
FIONBIO:<BR>+ case FIONREAD:<BR>+ case SIOCATMARK:<BR>+ if
(lpvInBuffer && cbInBuffer >= sizeof(u_long))
{<BR>+ ret =
rioctlsocket(<BR>+ (int)rs,<BR>+ dwIoControlCode,<BR>+ (u_long
*)lpvInBuffer<BR>+ );<BR>+ break;<BR>+ }<BR>+ default:<BR>+ ret
= SOCKET_ERROR;<BR>+ *lpErrno = WSAEINVAL;<BR>+ }<BR>+ if
(lpResultBuffer) {<BR>+ memcpy(lpvOutBuffer, lpResultBuffer,
*lpcbBytesReturned);<BR>+ free(lpResultBuffer);<BR>+ }<BR>+ <BR>+ return
ret;<BR>+}<BR>+<BR>+int WSPAPI<BR>+WSPAddressToString(<BR>+
__in LPSOCKADDR lpsaAddress,<BR>+
__in DWORD dwAddressLength,<BR>+
__in LPWSAPROTOCOL_INFOW lpProtocolInfo,<BR>+
__out LPWSTR lpszAddressString,<BR>+ __inout
LPDWORD lpdwAddressStringLength,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ int ret = 0;<BR>+ <BR>+ if
(lpProtocolInfo) {<BR>+ if (0 !=
memcmp(&lpProtocolInfo->ProviderId, &rsProviderGuid,
sizeof(rsProviderGuid))) {<BR>+ *lpErrno =
WSAEINVALIDPROVIDER;<BR>+ return
SOCKET_ERROR;<BR>+ }<BR>+ }<BR>+ if (SOCKET_ERROR == (ret =
WSAAddressToStringW(<BR>+ lpsaAddress,<BR>+ dwAddressLength,<BR>+ NULL,<BR>+ lpszAddressString,<BR>+ lpdwAddressStringLength<BR>+ )))<BR>+ *lpErrno
= WSAGetLastError();<BR>+<BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPAsyncSelect(<BR>+ __in SOCKET s,<BR>+
__in HWND hWnd,<BR>+ __in unsigned int
wMsg,<BR>+ __in long lEvent,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ *lpErrno = WSAEOPNOTSUPP;<BR>+ return
SOCKET_ERROR;<BR>+}<BR>+<BR>+int WSPAPI<BR>+WSPCancelBlockingCall(<BR>+
__out LPINT lpErrno<BR>+)<BR>+{<BR>+ *lpErrno =
WSAEOPNOTSUPP;<BR>+ return SOCKET_ERROR;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPDuplicateSocket(<BR>+ IN SOCKET
s,<BR>+ IN DWORD dwProcessId,<BR>+ OUT
LPWSAPROTOCOL_INFOW lpProtocolInfo,<BR>+ OUT LPINT
lpErrno<BR>+ )<BR>+{<BR>+ *lpErrno =
WSAEOPNOTSUPP;<BR>+ return SOCKET_ERROR;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPEnumNetworkEvents(<BR>+ IN SOCKET
s,<BR>+ IN WSAEVENT hEventObject,<BR>+
OUT LPWSANETWORKEVENTS lpNetworkEvents,<BR>+ OUT
LPINT lpErrno<BR>+ )<BR>+{<BR>+ *lpErrno =
WSAEOPNOTSUPP;<BR>+ return SOCKET_ERROR;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPEventSelect(<BR>+ __in SOCKET s,<BR>+
__in WSAEVENT hEventObject,<BR>+ __in long
lNetworkEvents,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ *lpErrno = WSAEOPNOTSUPP;<BR>+ return
SOCKET_ERROR;<BR>+}<BR>+<BR>+BOOL
WSPAPI<BR>+WSPGetOverlappedResult(<BR>+ IN SOCKET
s,<BR>+ IN LPWSAOVERLAPPED
lpOverlapped,<BR>+ OUT LPDWORD
lpcbTransfer,<BR>+ IN BOOL fWait,<BR>+
OUT LPDWORD lpdwFlags,<BR>+ OUT LPINT
lpErrno<BR>+ )<BR>+{<BR>+ *lpErrno =
WSAEOPNOTSUPP;<BR>+ return FALSE;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPGetPeerName(<BR>+ __in SOCKET
s,<BR>+ __out struct sockaddr* name,<BR>+
__inout LPINT namelen,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ if (SOCKET_ERROR == (ret = rgetpeername((int)rs, name,
namelen)))<BR>+ *lpErrno = WSAGetLastError();<BR>+<BR>+ return
ret;<BR>+}<BR>+<BR>+int WSPAPI<BR>+WSPGetSockName(<BR>+
__in SOCKET s,<BR>+ __out struct
sockaddr* name,<BR>+ __inout LPINT namelen,<BR>+
__out LPINT lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ if (SOCKET_ERROR == (ret = rgetsockname((int)rs, name,
namelen)))<BR>+ *lpErrno = WSAGetLastError();<BR>+<BR>+ return
ret;<BR>+}<BR>+<BR>+BOOL WSPAPI<BR>+WSPGetQOSByName(<BR>+
__in SOCKET s,<BR>+ __inout LPWSABUF
lpQOSName,<BR>+ __out LPQOS lpQOS,<BR>+
__out LPINT lpErrno<BR>+)<BR>+{<BR>+ *lpErrno =
WSAEOPNOTSUPP;<BR>+ return FALSE;<BR>+}<BR>+<BR>+SOCKET
WSPAPI<BR>+WSPJoinLeaf(<BR>+ __in SOCKET s,<BR>+
__in const struct sockaddr* name,<BR>+ __in int
namelen,<BR>+ __in LPWSABUF lpCallerData,<BR>+
__out LPWSABUF lpCalleeData,<BR>+ __in LPQOS
lpSQOS,<BR>+ __in LPQOS lpGQOS,<BR>+ __in
DWORD dwFlags,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ *lpErrno = WSAEOPNOTSUPP;<BR>+ return
INVALID_SOCKET;<BR>+}<BR>+<BR>+int WSPAPI<BR>+WSPRecvDisconnect(<BR>+
__in SOCKET s,<BR>+ __out LPWSABUF
lpInboundDisconnectData,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ return WSPShutdown(s, SD_RECEIVE, lpErrno); //
Ignore lpInboundDisconnectData<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPRecvFrom(<BR>+ __in SOCKET
s,<BR>+ __inout LPWSABUF lpBuffers,<BR>+
__in DWORD dwBufferCount,<BR>+
__out LPDWORD lpNumberOfBytesRecvd,<BR>+ __inout
LPDWORD lpFlags,<BR>+ __out struct sockaddr*
lpFrom,<BR>+ __inout LPINT lpFromlen,<BR>+
__in LPWSAOVERLAPPED lpOverlapped,<BR>+
__in LPWSAOVERLAPPED_COMPLETION_ROUTINE
lpCompletionRoutine,<BR>+ __in LPWSATHREADID
lpThreadId,<BR>+ __inout LPINT
lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR rs =
INVALID_SOCKET;<BR>+ DWORD
i;<BR>+ DWORD dwNumberOfBytesRecvd =
0;<BR>+ int len =
0;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ for (i = 0; i < dwBufferCount;
i++) {<BR>+ switch (len =
(int)rrecvfrom(<BR>+ (int)rs,<BR>+ lpBuffers[i].buf,<BR>+ lpBuffers[i].len,<BR>+ *lpFlags,<BR>+ lpFrom,<BR>+ lpFromlen<BR>+ ))
{<BR>+ case 0:<BR>+ goto
out;<BR>+ case -1:<BR>+ *lpErrno =
WSAGetLastError();<BR>+ ret =
SOCKET_ERROR;<BR>+ goto
out;<BR>+ default:<BR>+ dwNumberOfBytesRecvd +=
len;<BR>+ }<BR>+ }<BR>+<BR>+out:<BR>+ *lpNumberOfBytesRecvd
= dwNumberOfBytesRecvd;<BR>+ <BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPSendDisconnect(<BR>+ __in SOCKET s,<BR>+
__in LPWSABUF lpOutboundDisconnectData,<BR>+ __out LPINT
lpErrno<BR>+)<BR>+{<BR>+ return WSPShutdown(s, SD_SEND, lpErrno); // Ignore
lpOutboundDisconnectData<BR>+}<BR>+<BR>+int WSPAPI<BR>+WSPSendTo(<BR>+
__in SOCKET s,<BR>+ __in LPWSABUF
lpBuffers,<BR>+ __in DWORD dwBufferCount,<BR>+
__out LPDWORD lpNumberOfBytesSent,<BR>+ __in DWORD
dwFlags,<BR>+ __in const struct sockaddr* lpTo,<BR>+
__in int iTolen,<BR>+ __in LPWSAOVERLAPPED
lpOverlapped,<BR>+ __in LPWSAOVERLAPPED_COMPLETION_ROUTINE
lpCompletionRoutine,<BR>+ __in LPWSATHREADID
lpThreadId,<BR>+ __out LPINT lpErrno<BR>+)<BR>+{<BR>+ DWORD_PTR
rs = INVALID_SOCKET;<BR>+ DWORD
i;<BR>+ DWORD dwNumberOfBytesSent =
0;<BR>+ int
len;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(s, &rs,
lpErrno);<BR>+<BR>+ if (SOCKET_ERROR == ret)<BR>+ return
ret;<BR>+<BR>+ for (i = 0; i < dwBufferCount;
i++) {<BR>+ if (-1 == (len =
(int)rsendto(<BR>+ (int)rs,<BR>+ lpBuffers[i].buf,<BR>+ lpBuffers[i].len,<BR>+ dwFlags,<BR>+ lpTo,<BR>+ iTolen<BR>+ ))) {<BR>+ *lpErrno
= WSAGetLastError();<BR>+ ret =
SOCKET_ERROR;<BR>+ break;<BR>+ }<BR>+ else<BR>+ dwNumberOfBytesSent
+= len;<BR>+ }<BR>+ *lpNumberOfBytesSent =
dwNumberOfBytesSent;<BR>+ <BR>+ return ret;<BR>+}<BR>+<BR>+int
WSPAPI<BR>+WSPStringToAddress(<BR>+ __in LPWSTR
AddressString,<BR>+ __in INT
AddressFamily,<BR>+ __in LPWSAPROTOCOL_INFO
lpProtocolInfo,<BR>+ __out LPSOCKADDR
lpAddress,<BR>+ __inout LPINT lpAddressLength,<BR>+
__out LPINT lpErrno<BR>+)<BR>+{<BR>+ int ret =
0;<BR>+ <BR>+ if (AF_INET != AddressFamily && AddressFamily !=
AF_INET6) {<BR>+ *lpErrno = WSAEAFNOSUPPORT;<BR>+ return
SOCKET_ERROR;<BR>+ }<BR>+ <BR>+ if
(lpProtocolInfo) {<BR>+ if (0 !=
memcmp(&lpProtocolInfo->ProviderId, &rsProviderGuid,
sizeof(rsProviderGuid))) {<BR>+ *lpErrno =
WSAEINVALIDPROVIDER;<BR>+ return
SOCKET_ERROR;<BR>+ }<BR>+ }<BR>+ <BR>+ if
(SOCKET_ERROR == (ret =
WSAStringToAddressW(<BR>+ AddressString,<BR>+ AddressFamily,<BR>+ NULL,<BR>+ lpAddress,<BR>+ lpAddressLength<BR>+ )))<BR>+ *lpErrno
= WSAGetLastError();<BR>+<BR>+ return ret;<BR>+}<BR>+<BR>+/*<BR>+ *
Function: WSPStartup<BR>+ *<BR>+ * Description:<BR>+ * This
function initializes the base provider. <BR>+ */<BR>+#ifdef
__cplusplus<BR>+extern "C"
{<BR>+#endif<BR>+<BR>+__declspec(dllexport)<BR>+__checkReturn<BR>+int<BR>+WSPAPI<BR>+WSPStartup(<BR>+
__in WORD wVersion,<BR>+ __in LPWSPDATA
lpWSPData,<BR>+ __in LPWSAPROTOCOL_INFOW
lpProtocolInfo,<BR>+ __in WSPUPCALLTABLE
UpCallTable,<BR>+ __out LPWSPPROC_TABLE
lpProcTable<BR>+ )<BR>+{<BR>+ static
WSPDATA gWSPData;<BR>+ static
WSPPROC_TABLE gProcTable;<BR>+
int Error =
NO_ERROR,<BR>+ rc;<BR>+<BR>+TRACE("Requested
version = %d.%d", LOBYTE(wVersion), HIBYTE(wVersion));<BR>+<BR>+ /* Make
sure that the version requested is >= 2.2. The low byte is the
<BR>+ major version and the high byte is the minor version.
*/<BR>+ if( (LOBYTE(wVersion) < 2) || ((LOBYTE(wVersion) == 2)
&& (HIBYTE(wVersion) < 2)) )<BR>+ return
WSAVERNOTSUPPORTED;<BR>+<BR>+ EnterCriticalSection(
&gCriticalSection );<BR>+<BR>+ // The first time the
startup is called, create our heap and allocate some<BR>+
// data structures for tracking the LSP
providers<BR>+ if ( 0 == gStartupCount
)<BR>+ {<BR>+TRACE("Called 1st time => Initializing
ProtocolInfo...");<BR>+ /* Save the global WSPData
*/<BR>+ gWSPData.wVersion = MAKEWORD(2,
2);<BR>+ gWSPData.wHighVersion = MAKEWORD(2,
2);<BR>+ wcscpy_s( gWSPData.szDescription,
2*sizeof(gWSPData.szDescription), Description );<BR>+<BR>+ /* provide
Service provider's entry points in proc table */<BR>+ ZeroMemory(
&gProcTable, sizeof(gProcTable)
);<BR>+ gProcTable.lpWSPSocket =
WSPSocket;<BR>+ gProcTable.lpWSPBind =
WSPBind;<BR>+ gProcTable.lpWSPListen =
WSPListen;<BR>+ gProcTable.lpWSPAccept =
WSPAccept;<BR>+ gProcTable.lpWSPConnect =
WSPConnect;<BR>+ gProcTable.lpWSPShutdown =
WSPShutdown;<BR>+ gProcTable.lpWSPCloseSocket =
WSPCloseSocket;<BR>+ gProcTable.lpWSPRecv =
WSPRecv;<BR>+ gProcTable.lpWSPSend =
WSPSend;<BR>+ gProcTable.lpWSPGetSockOpt =
WSPGetSockOpt;<BR>+ gProcTable.lpWSPSetSockOpt =
WSPSetSockOpt;<BR>+ gProcTable.lpWSPSelect =
WSPSelect;<BR>+ gProcTable.lpWSPIoctl =
WSPIoctl;<BR>+ gProcTable.lpWSPCleanup =
WSPCleanup;<BR>+// Additional functions required for (base provider's)
WSPStartup:<BR>+ gProcTable.lpWSPAddressToString =
WSPAddressToString;<BR>+ gProcTable.lpWSPAsyncSelect =
WSPAsyncSelect;<BR>+ gProcTable.lpWSPCancelBlockingCall =
WSPCancelBlockingCall;<BR>+ gProcTable.lpWSPDuplicateSocket =
WSPDuplicateSocket;<BR>+ gProcTable.lpWSPEnumNetworkEvents =
WSPEnumNetworkEvents;<BR>+ gProcTable.lpWSPEventSelect =
WSPEventSelect;<BR>+ gProcTable.lpWSPGetOverlappedResult =
WSPGetOverlappedResult;<BR>+ gProcTable.lpWSPGetPeerName =
WSPGetPeerName;<BR>+ gProcTable.lpWSPGetSockName =
WSPGetSockName;<BR>+ gProcTable.lpWSPGetQOSByName =
WSPGetQOSByName;<BR>+ gProcTable.lpWSPJoinLeaf =
WSPJoinLeaf;<BR>+ gProcTable.lpWSPRecvDisconnect =
WSPRecvDisconnect;<BR>+ gProcTable.lpWSPRecvFrom =
WSPRecvFrom;<BR>+ gProcTable.lpWSPSendDisconnect =
WSPSendDisconnect;<BR>+ gProcTable.lpWSPSendTo =
WSPSendTo;<BR>+ gProcTable.lpWSPStringToAddress =
(LPWSPSTRINGTOADDRESS)WSPStringToAddress;<BR>+ <BR>+ gProtocolInfo
= *lpProtocolInfo;<BR>+ }<BR>+
gStartupCount++;<BR>+TRACE("StartupCount incremented to %d",
gStartupCount);<BR>+<BR>+ LeaveCriticalSection(
&gCriticalSection );<BR>+<BR>+ /* Set the return parameters
*/<BR>+ *lpWSPData = gWSPData;<BR>+ *lpProcTable =
gProcTable;<BR>+ <BR>+ /* store the upcall function table
*/<BR>+ gMainUpCallTable =
UpCallTable;<BR>+<BR>+ return Error;<BR>+}<BR>+<BR>+#ifdef
__cplusplus<BR>+}<BR>+#endif<BR>+<BR> BOOL WINAPI DllMain(HINSTANCE
hInstance, DWORD dwReason, LPVOID
lpReserved)<BR> {<BR> UNREFERENCED_PARAMETER(hInstance);<BR>@@
-44,10 +1195,28 @@<BR> if (heap == NULL)
{<BR> return
FALSE;<BR> }<BR>- InitializeCriticalSection(&lock);<BR>+ fastlock_init(&lock);<BR>+ fastlock_init(&mut);<BR>+ fastlock_init(&TraceLock);<BR>+<BR>+ //<BR>+ //
Initialize some critical section objects
<BR>+ //<BR>+ __try<BR>+ {<BR>+ InitializeCriticalSection(
&gCriticalSection );<BR>+ }<BR>+ __except(
EXCEPTION_EXECUTE_HANDLER )<BR>+ {<BR>+ goto
cleanup;<BR>+ }<BR> break;<BR> case
DLL_PROCESS_DETACH:<BR>- DeleteCriticalSection(&lock);<BR>+ gDetached
= TRUE;<BR>+ DeleteCriticalSection( &gCriticalSection
);<BR>+ fastlock_destroy(&mut);<BR>+ fastlock_destroy(&lock);<BR>+ fastlock_destroy(&TraceLock);<BR> HeapDestroy(heap);<BR> break;<BR> default:<BR>@@
-55,4 +1224,8 @@<BR> }<BR> <BR> return
TRUE;<BR>+<BR>+cleanup:<BR>+<BR>+ return
FALSE;<BR> }<BR>Index:
ulp/librdmacm/src/indexer.cpp<BR>===================================================================<BR>---
ulp/librdmacm/src/indexer.cpp (revision 0)<BR>+++
ulp/librdmacm/src/indexer.cpp (working copy)<BR>@@ -0,0 +1,174
@@<BR>+/*<BR>+ * Copyright (c) 2011 Intel Corporation. All rights
reserved.<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All rights
reserved.<BR>+ *<BR>+ * This software is available to you under the BSD license
below:<BR>+ *<BR>+ * Redistribution and use in source
and binary forms, with or<BR>+ * without modification,
are permitted provided that the following<BR>+ *
conditions are met:<BR>+ *<BR>+ * -
Redistributions of source code must retain the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer.<BR>+ *<BR>+ * - Redistributions in
binary form must reproduce the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer in the documentation and/or other materials<BR>+
* provided with the distribution.<BR>+
*<BR>+ * - Neither the name Oce Printing Systems
GmbH nor the names<BR>+ * of the
authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR>+ */<BR>+<BR>+#if HAVE_CONFIG_H<BR>+# include
<config.h><BR>+#endif /* HAVE_CONFIG_H */<BR>+<BR>+#include
<windows.h><BR>+<BR>+#include <sys/types.h><BR>+#include
<stdlib.h><BR>+<BR>+#include "indexer.h"<BR>+#include
"cma.h"<BR>+<BR>+/*<BR>+ * Indexer - to find a structure given an index<BR>+
*<BR>+ * We store pointers using a double lookup and return an index to the<BR>+
* user which is then used to retrieve the pointer. The upper bits of<BR>+
* the index are itself an index into an array of memory allocations.<BR>+ * The
lower bits specify the offset into the allocated memory where<BR>+ * the pointer
is stored.<BR>+ *<BR>+ * This allows us to adjust the number of pointers stored
by the index<BR>+ * list without taking a lock during data lookups.<BR>+
*/<BR>+<BR>+static int idx_grow(struct indexer *idx)<BR>+{<BR>+ union
idx_entry *entry;<BR>+ int i, start_index;<BR>+<BR>+ if (idx->size
>= IDX_ARRAY_SIZE)<BR>+ goto
nomem;<BR>+<BR>+ idx->array[idx->size] = (union idx_entry
*)calloc(IDX_ENTRY_SIZE, sizeof(union idx_entry));<BR>+ if
(!idx->array[idx->size])<BR>+ goto nomem;<BR>+<BR>+ entry
= idx->array[idx->size];<BR>+ start_index = idx->size <<
IDX_ENTRY_BITS;<BR>+ entry[IDX_ENTRY_SIZE - 1].next =
idx->free_list;<BR>+<BR>+ for (i = IDX_ENTRY_SIZE - 2; i >= 0;
i--)<BR>+ entry[i].next = start_index + i + 1;<BR>+<BR>+ /*
Index 0 is reserved */<BR>+ if (start_index ==
0)<BR>+ start_index++;<BR>+ idx->free_list =
start_index;<BR>+ idx->size++;<BR>+ return
start_index;<BR>+<BR>+nomem:<BR>+ errno = ENOMEM;<BR>+ return
-1;<BR>+}<BR>+<BR>+int idx_insert(struct indexer *idx, void
*item)<BR>+{<BR>+ union idx_entry *entry;<BR>+ int
index;<BR>+<BR>+ if ((index = idx->free_list) == 0) {<BR>+ if
((index = idx_grow(idx)) <= 0)<BR>+ return
index;<BR>+ }<BR>+<BR>+ entry =
idx->array[idx_array_index(index)];<BR>+ idx->free_list =
entry[idx_entry_index(index)].next;<BR>+ entry[idx_entry_index(index)].item
= item;<BR>+ return index;<BR>+}<BR>+<BR>+void *idx_remove(struct indexer
*idx, int index)<BR>+{<BR>+ union idx_entry *entry;<BR>+ void
*item;<BR>+<BR>+ entry =
idx->array[idx_array_index(index)];<BR>+ item =
entry[idx_entry_index(index)].item;<BR>+ entry[idx_entry_index(index)].next
= idx->free_list;<BR>+ idx->free_list = index;<BR>+ return
item;<BR>+}<BR>+<BR>+void idx_replace(struct indexer *idx, int index, void
*item)<BR>+{<BR>+ union idx_entry *entry;<BR>+<BR>+ entry =
idx->array[idx_array_index(index)];<BR>+ entry[idx_entry_index(index)].item
= item;<BR>+}<BR>+<BR>+<BR>+static int idm_grow(struct index_map *idm, int
index)<BR>+{<BR>+ idm->array[idx_array_index(index)] = (void
**)calloc(IDX_ENTRY_SIZE, sizeof(void *));<BR>+ if
(!idm->array[idx_array_index(index)])<BR>+ goto
nomem;<BR>+<BR>+ return index;<BR>+<BR>+nomem:<BR>+ errno =
ENOMEM;<BR>+ return -1;<BR>+}<BR>+<BR>+int idm_set(struct index_map *idm,
int index, void *item)<BR>+{<BR>+ void **entry;<BR>+<BR>+ if (index
> IDX_MAX_INDEX) {<BR>+ errno = ENOMEM;<BR>+ return
-1;<BR>+ }<BR>+<BR>+ if (!idm->array[idx_array_index(index)])
{<BR>+ if (idm_grow(idm, index) < 0)<BR>+ return
-1;<BR>+ }<BR>+<BR>+ entry =
idm->array[idx_array_index(index)];<BR>+ entry[idx_entry_index(index)] =
item;<BR>+ return index;<BR>+}<BR>+<BR>+void *idm_clear(struct index_map
*idm, int index)<BR>+{<BR>+ void **entry;<BR>+ void
*item;<BR>+<BR>+ entry =
idm->array[idx_array_index(index)];<BR>+ item =
entry[idx_entry_index(index)];<BR>+ entry[idx_entry_index(index)] =
NULL;<BR>+ return item;<BR>+}<BR>Index:
ulp/librdmacm/src/indexer.h<BR>===================================================================<BR>---
ulp/librdmacm/src/indexer.h (revision 0)<BR>+++
ulp/librdmacm/src/indexer.h (working copy)<BR>@@ -0,0 +1,101 @@<BR>+/*<BR>+
* Copyright (c) 2011 Intel Corporation. All rights reserved.<BR>+ *<BR>+ *
This software is available to you under a choice of one of two<BR>+ *
licenses. You may choose to be licensed under the terms of the GNU<BR>+ *
General Public License (GPL) Version 2, available from the file<BR>+ * COPYING
in the main directory of this source tree, or the<BR>+ * OpenIB.org BSD license
below:<BR>+ *<BR>+ * Redistribution and use in source
and binary forms, with or<BR>+ * without modification,
are permitted provided that the following<BR>+ *
conditions are met:<BR>+ *<BR>+ * -
Redistributions of source code must retain the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer.<BR>+ *<BR>+ * - Redistributions in
binary form must reproduce the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer in the documentation and/or other materials<BR>+
* provided with the distribution.<BR>+
*<BR>+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,<BR>+ *
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF<BR>+ *
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND<BR>+ * NONINFRINGEMENT. IN
NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS<BR>+ * BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN<BR>+ * ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN<BR>+ * CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE<BR>+ * SOFTWARE.<BR>+ *<BR>+ */<BR>+<BR>+#if
HAVE_CONFIG_H<BR>+# include <config.h><BR>+#endif /* HAVE_CONFIG_H
*/<BR>+<BR>+#include <sys/types.h><BR>+<BR>+/*<BR>+ * Indexer - to find a
structure given an index. Synchronization<BR>+ * must be provided by the
caller. Caller must initialize the<BR>+ * indexer by setting free_list and
size to 0.<BR>+ */<BR>+<BR>+union idx_entry {<BR>+ void
*item;<BR>+ int next;<BR>+};<BR>+<BR>+#define IDX_INDEX_BITS
16<BR>+#define IDX_ENTRY_BITS 10<BR>+#define IDX_ENTRY_SIZE (1 <<
IDX_ENTRY_BITS)<BR>+#define IDX_ARRAY_SIZE (1 << (IDX_INDEX_BITS -
IDX_ENTRY_BITS))<BR>+#define IDX_MAX_INDEX ((1 << IDX_INDEX_BITS) -
1)<BR>+<BR>+struct indexer<BR>+{<BR>+ union idx_entry
*array[IDX_ARRAY_SIZE];<BR>+ int
free_list;<BR>+ int size;<BR>+};<BR>+<BR>+#define
idx_array_index(index) (index >> IDX_ENTRY_BITS)<BR>+#define
idx_entry_index(index) (index & (IDX_ENTRY_SIZE - 1))<BR>+<BR>+int
idx_insert(struct indexer *idx, void *item);<BR>+void *idx_remove(struct indexer
*idx, int index);<BR>+void idx_replace(struct indexer *idx, int index, void
*item);<BR>+<BR>+static inline void *idx_at(struct indexer *idx, int
index)<BR>+{<BR>+ return (idx->array[idx_array_index(index)] +
idx_entry_index(index))->item;<BR>+}<BR>+<BR>+/*<BR>+ * Index map -
associates a structure with an index. Synchronization<BR>+ * must be
provided by the caller. Caller must initialize the<BR>+ * index map by
setting it to 0.<BR>+ */<BR>+<BR>+struct index_map<BR>+{<BR>+ void
**array[IDX_ARRAY_SIZE];<BR>+};<BR>+<BR>+int idm_set(struct index_map *idm, int
index, void *item);<BR>+void *idm_clear(struct index_map *idm, int
index);<BR>+<BR>+static inline void *idm_at(struct index_map *idm, int
index)<BR>+{<BR>+ void **entry;<BR>+ entry =
idm->array[idx_array_index(index)];<BR>+ return
entry[idx_entry_index(index)];<BR>+}<BR>+<BR>+static inline void
*idm_lookup(struct index_map *idm, int index)<BR>+{<BR>+ return ((index
<= IDX_MAX_INDEX) && idm->array[idx_array_index(index)])
?<BR>+ idm_at(idm, index) : NULL;<BR>+}<BR>Index:
ulp/librdmacm/src/openib_osd.h<BR>===================================================================<BR>---
ulp/librdmacm/src/openib_osd.h (revision 0)<BR>+++
ulp/librdmacm/src/openib_osd.h (working copy)<BR>@@ -0,0 +1,47
@@<BR>+#ifndef OPENIB_OSD_H<BR>+#define OPENIB_OSD_H<BR>+<BR>+#if
defined(FD_SETSIZE) && FD_SETSIZE != 1024<BR>+#undef
FD_SETSIZE<BR>+#endif<BR>+#define FD_SETSIZE 1024 /* Set before including
winsock2 - see select help */<BR>+<BR>+#include <winsock2.h><BR>+#include
<ws2tcpip.h><BR>+#include <io.h><BR>+#include
<fcntl.h><BR>+<BR>+#define container_of CONTAINING_RECORD<BR>+#define
offsetof FIELD_OFFSET<BR>+#define
ssize_t SSIZE_T<BR>+<BR>+#define
__thread /*???*/<BR>+<BR>+#define
ntohll _byteswap_uint64<BR>+#define
htonll _byteswap_uint64<BR>+<BR>+#define
SHUT_RD SD_RECEIVE<BR>+#define
SHUT_WR SD_SEND<BR>+#define
SHUT_RDWR SD_BOTH<BR>+<BR>+#define
O_NONBLOCK 0x4000<BR>+<BR>+/* allow casting to WSABUF */<BR>+struct
iovec<BR>+{<BR>+ u_long
iov_len;<BR>+ char FAR*
iov_base;<BR>+};<BR>+<BR>+struct
msghdr<BR>+{<BR>+ void
*msg_name; // optional
address<BR>+ socklen_t
msg_namelen; // size of address<BR>+ struct iovec
*msg_iov; // scatter/gather
array<BR>+ int
msg_iovlen; // members in
msg_iov<BR>+ void
*msg_control; // ancillary data, see
below<BR>+ socklen_t msg_controllen; // ancillary
data buffer
len<BR>+ int
msg_flags; // flags on received
message<BR>+};<BR>+<BR>+#endif // OPENIB_OSD_H<BR>Index:
ulp/librdmacm/src/rsocket.cpp<BR>===================================================================<BR>---
ulp/librdmacm/src/rsocket.cpp (revision 0)<BR>+++
ulp/librdmacm/src/rsocket.cpp (working copy)<BR>@@ -0,0 +1,2334
@@<BR>+/*<BR>+ * Copyright (c) 2008-2012 Intel Corporation. All rights
reserved.<BR>+ * Copyright (c) 2012 Oce Printing Systems GmbH. All rights
reserved.<BR>+ *<BR>+ * This software is available to you under the BSD license
below:<BR>+ *<BR>+ * Redistribution and use in source
and binary forms, with or<BR>+ * without modification,
are permitted provided that the following<BR>+ *
conditions are met:<BR>+ *<BR>+ * -
Redistributions of source code must retain the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer.<BR>+ *<BR>+ * - Redistributions in
binary form must reproduce the above<BR>+
* copyright notice, this list of
conditions and the following<BR>+ *
disclaimer in the documentation and/or other materials<BR>+
* provided with the distribution.<BR>+
*<BR>+ * - Neither the name Oce Printing Systems
GmbH nor the names<BR>+ * of the
authors may be used to endorse or promote products<BR>+
* derived from this software without
specific prior written<BR>+ *
permission.<BR>+ *<BR>+ * THIS SOFTWARE IS PROVIDED “AS IS” AND ANY
EXPRESS OR IMPLIED<BR>+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
WARRANTIES OF<BR>+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
AND<BR>+ * NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS<BR>+ *
OR CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT,<BR>+ * INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL<BR>+ * DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF<BR>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS;<BR>+ * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF<BR>+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT<BR>+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF<BR>+ * THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY<BR>+ * OF SUCH DAMAGE.
<BR>+ */<BR>+<BR>+#if HAVE_CONFIG_H<BR>+# include
<config.h><BR>+#endif /* HAVE_CONFIG_H */<BR>+<BR>+#include
<sys/types.h><BR>+<BR>+#include <windows.h><BR>+#include
"openib_osd.h"<BR>+#include <sys/time.h><BR>+#include
<stdarg.h><BR>+#include <netdb.h><BR>+#include
<unistd.h><BR>+#include <_fcntl.h><BR>+#include
<stdio.h><BR>+#include <string.h><BR>+#include
<_errno.h><BR>+#include <complib/cl_byteswap.h><BR>+#include
<rdma/rdma_cma.h><BR>+#include <rdma/rdma_verbs.h><BR>+#include
<rdma/rwinsock.h><BR>+#include <rdma/rsocket.h><BR>+#include
"cma.h"<BR>+#include "indexer.h"<BR>+#include "../../../etc/user/gtod.c" //
getimeofday()<BR>+<BR>+#define RS_OLAP_START_SIZE 2048<BR>+#define
RS_MAX_TRANSFER 65536<BR>+#define RS_MAX_BACKLOG 256<BR>+#define
RS_QP_MAX_SIZE 0xFFFE<BR>+#define RS_QP_CTRL_SIZE 4<BR>+#define RS_CONN_RETRIES
6<BR>+#define RS_SGL_SIZE 2<BR>+<BR>+static struct index_map idm;<BR>+static
uint16_t def_inline = 64;<BR>+static uint16_t def_sqsize = 384;<BR>+static
uint16_t def_rqsize = 384;<BR>+static uint32_t def_mem = (1 <<
17);<BR>+static uint32_t def_wmem = (1 << 17);<BR>+static uint32_t
polling_time = 10;<BR>+<BR>+extern fastlock_t mut;<BR>+extern
BOOL gDetached; // Indicates if process
is detaching from DLL<BR>+extern CRITICAL_SECTION gCriticalSection;
// Critical section for initialization and <BR>+extern
WSPUPCALLTABLE gMainUpCallTable; // Upcall functions
given to us by Winsock<BR>+extern WSAPROTOCOL_INFOW
gProtocolInfo;<BR>+<BR>+/*<BR>+ * Immediate data format is determined by the
upper bits<BR>+ * bit 31: message type, 0 - data, 1 - control<BR>+ * bit 30:
buffers updated, 0 - target, 1 - direct-receive<BR>+ * bit 29: more data, 0 -
end of transfer, 1 - more data available<BR>+ *<BR>+ * for data transfers:<BR>+
* bits [28:0]: bytes transfered, 0 = 1 GB<BR>+ * for control messages:<BR>+ *
bits [28-0]: receive credits granted<BR>+ */<BR>+<BR>+enum
{<BR>+ RS_OP_DATA,<BR>+ RS_OP_RSVD_DATA_MORE,<BR>+ RS_OP_RSVD_DRA,<BR>+ RS_OP_RSVD_DRA_MORE,<BR>+ RS_OP_SGL,<BR>+ RS_OP_RSVD,<BR>+ RS_OP_RSVD_DRA_SGL,<BR>+ RS_OP_CTRL<BR>+};<BR>+#define
rs_msg_set(op, data) ((op << 29) | (uint32_t) (data))<BR>+#define
rs_msg_op(imm_data) (imm_data >> 29)<BR>+#define
rs_msg_data(imm_data) (imm_data & 0x1FFFFFFF)<BR>+<BR>+enum
{<BR>+ RS_CTRL_DISCONNECT,<BR>+ RS_CTRL_SHUTDOWN<BR>+};<BR>+<BR>+struct
rs_msg {<BR>+ uint32_t op;<BR>+ uint32_t data;<BR>+};<BR>+<BR>+struct
rs_sge {<BR>+ uint64_t addr;<BR>+ uint32_t key;<BR>+ uint32_t
length;<BR>+};<BR>+<BR>+#define RS_MIN_INLINE (sizeof(struct
rs_sge))<BR>+#define rs_host_is_net() (1 == htonl(1))<BR>+#define
RS_CONN_FLAG_NET 1<BR>+<BR>+struct rs_conn_data
{<BR>+ uint8_t
version;<BR>+ uint8_t
flags;<BR>+ uint16_t credits;<BR>+ uint32_t
reserved2;<BR>+ struct rs_sge target_sgl;<BR>+ struct
rs_sge data_buf;<BR>+};<BR>+<BR>+#define RS_RECV_WR_ID (~((uint64_t)
0))<BR>+<BR>+/*<BR>+ * rsocket states are ordered as passive, connecting,
connected, disconnected.<BR>+ */<BR>+enum rs_state
{<BR>+ rs_init,<BR>+ rs_bound
= 0x0001,<BR>+ rs_listening
= 0x0002,<BR>+ rs_opening
= 0x0004,<BR>+ rs_resolving_addr =
rs_opening | 0x0010,<BR>+ rs_resolving_route = rs_opening
| 0x0020,<BR>+ rs_connecting =
rs_opening |
0x0040,<BR>+ rs_accepting = rs_opening
| 0x0080,<BR>+ rs_connected
= 0x0100,<BR>+ rs_connect_wr
=
0x0200,<BR>+ rs_connect_rd
=
0x0400,<BR>+ rs_connect_rdwr = rs_connected |
rs_connect_rd | rs_connect_wr,<BR>+ rs_connect_error
=
0x0800,<BR>+ rs_disconnected
= 0x1000,<BR>+ rs_error
= 0x2000,<BR>+};<BR>+<BR>+#define RS_OPT_SWAP_SGL
1<BR>+<BR>+struct rsocket {<BR>+ struct rdma_cm_id
*cm_id;<BR>+ fastlock_t slock;<BR>+ fastlock_t
rlock;<BR>+ fastlock_t
cq_lock;<BR>+ fastlock_t
cq_wait_lock;<BR>+<BR>+ int
opts;<BR>+ long fd_flags;<BR>+ uint64_t
so_opts;<BR>+ uint64_t tcp_opts;<BR>+ uint64_t
ipv6_opts;<BR>+ int
state;<BR>+ int cq_armed;<BR>+ int
retries;<BR>+ int err;<BR>+ int
index;<BR>+ int
ctrl_avail;<BR>+ int
sqe_avail;<BR>+ int
sbuf_bytes_avail;<BR>+ uint16_t
sseq_no;<BR>+ uint16_t
sseq_comp;<BR>+ uint16_t
sq_size;<BR>+ uint16_t
sq_inline;<BR>+<BR>+ uint16_t
rq_size;<BR>+ uint16_t rseq_no;<BR>+ uint16_t
rseq_comp;<BR>+ int
rbuf_bytes_avail;<BR>+ int
rbuf_free_offset;<BR>+ int
rbuf_offset;<BR>+ int
rmsg_head;<BR>+ int rmsg_tail;<BR>+ struct
rs_msg *rmsg;<BR>+<BR>+ int
remote_sge;<BR>+ struct rs_sge
remote_sgl;<BR>+<BR>+ struct ibv_mr
*target_mr;<BR>+ int target_sge;<BR>+ volatile
struct rs_sge
target_sgl[RS_SGL_SIZE];<BR>+<BR>+ uint32_t
rbuf_size;<BR>+ struct ibv_mr
*rmr;<BR>+ uint8_t
*rbuf;<BR>+<BR>+ uint32_t sbuf_size;<BR>+ struct
ibv_mr *smr;<BR>+ struct ibv_sge
ssgl[2];<BR>+ uint8_t *sbuf;<BR>+};<BR>+<BR>+static void
rs_configure(void)<BR>+{<BR>+ FILE *f;<BR>+ static int
init;<BR>+<BR>+ if
(init)<BR>+ return;<BR>+<BR>+ fastlock_acquire(&mut);<BR>+ if
(init)<BR>+ goto out;<BR>+<BR>+ if ((f = fopen(RS_CONF_DIR
"/polling_time", "r"))) {<BR>+ fscanf(f, "%u",
&polling_time);<BR>+ fclose(f);<BR>+ }<BR>+<BR>+ if ((f
= fopen(RS_CONF_DIR "/inline_default", "r"))) {<BR>+ fscanf(f, "%hu",
&def_inline);<BR>+ fclose(f);<BR>+<BR>+ if (def_inline
< RS_MIN_INLINE)<BR>+ def_inline =
RS_MIN_INLINE;<BR>+ }<BR>+<BR>+ if ((f = fopen(RS_CONF_DIR
"/sqsize_default", "r"))) {<BR>+ fscanf(f, "%hu",
&def_sqsize);<BR>+ fclose(f);<BR>+ }<BR>+<BR>+ if ((f =
fopen(RS_CONF_DIR "/rqsize_default", "r"))) {<BR>+ fscanf(f, "%hu",
&def_rqsize);<BR>+ fclose(f);<BR>+ }<BR>+<BR>+ if ((f =
fopen(RS_CONF_DIR "/mem_default", "r"))) {<BR>+ fscanf(f, "%u",
&def_mem);<BR>+ fclose(f);<BR>+<BR>+ if (def_mem <
1)<BR>+ def_mem = 1;<BR>+ }<BR>+<BR>+ if ((f =
fopen(RS_CONF_DIR "/wmem_default", "r"))) {<BR>+ fscanf(f, "%u",
&def_wmem);<BR>+ fclose(f);<BR>+<BR>+ if (def_wmem
< 1)<BR>+ def_wmem = 1;<BR>+ }<BR>+ init =
1;<BR>+out:<BR>+ fastlock_release(&mut);<BR>+}<BR>+<BR>+static int
rs_insert(struct rsocket
*rs)<BR>+{<BR>+ fastlock_acquire(&mut);<BR>+ rs->index =
idm_set(&idm, ((int)rs->cm_id->channel->channel.Event >>
2)/*fd*/, rs);<BR>+ fastlock_release(&mut);<BR>+ return
rs->index;<BR>+}<BR>+<BR>+static void rs_remove(struct rsocket
*rs)<BR>+{<BR>+ fastlock_acquire(&mut);<BR>+ idm_clear(&idm,
rs->index);<BR>+ fastlock_release(&mut);<BR>+}<BR>+<BR>+static
struct rsocket *rs_alloc(struct rsocket *inherited_rs)<BR>+{<BR>+ struct
rsocket *rs;<BR>+<BR>+ rs = (struct rsocket *)calloc(1, sizeof
*rs);<BR>+ if (!rs)<BR>+ return
NULL;<BR>+<BR>+ rs->index = -1;<BR>+ if (inherited_rs)
{<BR>+ rs->sbuf_size =
inherited_rs->sbuf_size;<BR>+ rs->rbuf_size =
inherited_rs->rbuf_size;<BR>+ rs->sq_inline =
inherited_rs->sq_inline;<BR>+ rs->sq_size =
inherited_rs->sq_size;<BR>+ rs->rq_size =
inherited_rs->rq_size;<BR>+ rs->ctrl_avail =
inherited_rs->ctrl_avail;<BR>+ } else {<BR>+ rs->sbuf_size
= def_wmem;<BR>+ rs->rbuf_size =
def_mem;<BR>+ rs->sq_inline =
def_inline;<BR>+ rs->sq_size =
def_sqsize;<BR>+ rs->rq_size =
def_rqsize;<BR>+ rs->ctrl_avail =
RS_QP_CTRL_SIZE;<BR>+ }<BR>+ fastlock_init(&rs->slock);<BR>+ fastlock_init(&rs->rlock);<BR>+ fastlock_init(&rs->cq_lock);<BR>+ fastlock_init(&rs->cq_wait_lock);<BR>+ return
rs;<BR>+}<BR>+<BR>+static int rs_set_nonblocking(struct rsocket *rs, long
arg)<BR>+{<BR>+ if
(rs->cm_id->recv_cq_channel)<BR>+ rs->cm_id->recv_cq_channel->comp_channel.Milliseconds
= (arg == O_NONBLOCK ? 0 : INFINITE);<BR>+<BR>+ if (rs->state <
rs_connected)<BR>+ rs->cm_id->channel->channel.Milliseconds
= (arg == O_NONBLOCK ? 0 : INFINITE);<BR>+<BR>+ return
0;<BR>+}<BR>+<BR>+static void rs_set_qp_size(struct rsocket
*rs)<BR>+{<BR>+ uint16_t max_size;<BR>+<BR>+ max_size =
min(ucma_max_qpsize(rs->cm_id), RS_QP_MAX_SIZE);<BR>+<BR>+ if
(rs->sq_size > max_size)<BR>+ rs->sq_size =
max_size;<BR>+ else if (rs->sq_size <
2)<BR>+ rs->sq_size = 2;<BR>+ if (rs->sq_size <=
(RS_QP_CTRL_SIZE << 2))<BR>+ rs->ctrl_avail =
1;<BR>+<BR>+ if (rs->rq_size >
max_size)<BR>+ rs->rq_size = max_size;<BR>+ else if
(rs->rq_size < 2)<BR>+ rs->rq_size =
2;<BR>+}<BR>+<BR>+static int rs_init_bufs(struct rsocket
*rs)<BR>+{<BR>+ rs->rmsg = (struct rs_msg *)calloc(rs->rq_size + 1,
sizeof(*rs->rmsg));<BR>+ if (!rs->rmsg)<BR>+ return
-1;<BR>+<BR>+ rs->sbuf = (uint8_t *)calloc(rs->sbuf_size,
sizeof(*rs->sbuf));<BR>+ if (!rs->sbuf)<BR>+ return
-1;<BR>+<BR>+ rs->smr = rdma_reg_msgs(rs->cm_id, rs->sbuf,
rs->sbuf_size);<BR>+ if (!rs->smr)<BR>+ return
-1;<BR>+<BR>+ rs->target_mr = rdma_reg_write(rs->cm_id, (void *)
rs->target_sgl,<BR>+
sizeof(rs->target_sgl));<BR>+ if
(!rs->target_mr)<BR>+ return -1;<BR>+<BR>+ rs->rbuf =
(uint8_t *)calloc(rs->rbuf_size, sizeof(*rs->rbuf));<BR>+ if
(!rs->rbuf)<BR>+ return -1;<BR>+<BR>+ rs->rmr =
rdma_reg_write(rs->cm_id, rs->rbuf, rs->rbuf_size);<BR>+ if
(!rs->rmr)<BR>+ return -1;<BR>+<BR>+ rs->ssgl[0].addr =
rs->ssgl[1].addr = (uintptr_t) rs->sbuf;<BR>+ rs->sbuf_bytes_avail
= rs->sbuf_size;<BR>+ rs->ssgl[0].lkey = rs->ssgl[1].lkey =
rs->smr->lkey;<BR>+<BR>+ rs->rbuf_free_offset = rs->rbuf_size
>> 1;<BR>+ rs->rbuf_bytes_avail = rs->rbuf_size >>
1;<BR>+ rs->sqe_avail = rs->sq_size -
rs->ctrl_avail;<BR>+ rs->rseq_comp = rs->rq_size >>
1;<BR>+ return 0;<BR>+}<BR>+<BR>+static int rs_create_cq(struct rsocket
*rs)<BR>+{<BR>+ rs->cm_id->recv_cq_channel =
ibv_create_comp_channel(rs->cm_id->verbs);<BR>+ if
(!rs->cm_id->recv_cq_channel)<BR>+ return
-1;<BR>+<BR>+ rs->cm_id->recv_cq =
ibv_create_cq(rs->cm_id->verbs, rs->sq_size +
rs->rq_size,<BR>+ rs->cm_id,
rs->cm_id->recv_cq_channel, 0);<BR>+ if
(!rs->cm_id->recv_cq)<BR>+ goto err1;<BR>+<BR>+ if
(rs->fd_flags & O_NONBLOCK) {<BR>+ if (rs_set_nonblocking(rs,
O_NONBLOCK))<BR>+ goto
err2;<BR>+ }<BR>+<BR>+ rs->cm_id->send_cq_channel =
rs->cm_id->recv_cq_channel;<BR>+ rs->cm_id->send_cq =
rs->cm_id->recv_cq;<BR>+ return
0;<BR>+<BR>+err2:<BR>+ ibv_destroy_cq(rs->cm_id->recv_cq);<BR>+ rs->cm_id->recv_cq
=
NULL;<BR>+err1:<BR>+ ibv_destroy_comp_channel(rs->cm_id->recv_cq_channel);<BR>+ rs->cm_id->recv_cq_channel
= NULL;<BR>+ return -1;<BR>+}<BR>+<BR>+static __inline
int<BR>+rs_post_recv(struct rsocket *rs)<BR>+{<BR>+ struct ibv_recv_wr wr,
*bad;<BR>+<BR>+ wr.wr_id = RS_RECV_WR_ID;<BR>+ wr.next =
NULL;<BR>+ wr.sg_list = NULL;<BR>+ wr.num_sge =
0;<BR>+<BR>+ return rdma_seterrno(ibv_post_recv(rs->cm_id->qp,
&wr, &bad));<BR>+}<BR>+<BR>+static int rs_create_ep(struct rsocket
*rs)<BR>+{<BR>+ struct ibv_qp_init_attr qp_attr;<BR>+ int i,
ret;<BR>+<BR>+ rs_set_qp_size(rs);<BR>+<BR>+ ret =
rs_create_cq(rs);<BR>+ if (ret)<BR>+ return
ret;<BR>+<BR>+ memset(&qp_attr, 0, sizeof
qp_attr);<BR>+ qp_attr.qp_context = rs;<BR>+ qp_attr.send_cq =
rs->cm_id->send_cq;<BR>+ qp_attr.recv_cq =
rs->cm_id->recv_cq;<BR>+ qp_attr.qp_type =
IBV_QPT_RC;<BR>+ qp_attr.sq_sig_all = 1;<BR>+ qp_attr.cap.max_send_wr
= rs->sq_size;<BR>+ qp_attr.cap.max_recv_wr =
rs->rq_size;<BR>+ qp_attr.cap.max_send_sge =
2;<BR>+ qp_attr.cap.max_recv_sge = 1;<BR>+ qp_attr.cap.max_inline_data
= rs->sq_inline;<BR>+<BR>+ ret = rdma_create_qp(rs->cm_id, NULL,
&qp_attr);<BR>+ if (ret)<BR>+ return ret;<BR>+<BR>+ ret
= rs_init_bufs(rs); <BR>+ if (ret)<BR>+ return
ret;<BR>+<BR>+ for (i = 0; i < rs->rq_size; i++)
{<BR>+ ret = rs_post_recv(rs);<BR>+ if
(ret)<BR>+ return ret;<BR>+ }<BR>+ return
0;<BR>+}<BR>+<BR>+static void rs_free(struct rsocket *rs)<BR>+{<BR>+ if
(rs->index >= 0)<BR>+ rs_remove(rs);<BR>+<BR>+ if
(rs->rmsg)<BR>+ free(rs->rmsg);<BR>+<BR>+ if (rs->sbuf)
{<BR>+ if
(rs->smr)<BR>+ rdma_dereg_mr(rs->smr);<BR>+ free(rs->sbuf);<BR>+ }<BR>+<BR>+ if
(rs->rbuf) {<BR>+ if
(rs->rmr)<BR>+ rdma_dereg_mr(rs->rmr);<BR>+ free(rs->rbuf);<BR>+ }<BR>+<BR>+ if
(rs->target_mr)<BR>+ rdma_dereg_mr(rs->target_mr);<BR>+<BR>+ if
(rs->cm_id) {<BR>+ if
(rs->cm_id->qp)<BR>+ rdma_destroy_qp(rs->cm_id);<BR>+ rdma_destroy_id(rs->cm_id);<BR>+ }<BR>+<BR>+ fastlock_destroy(&rs->cq_wait_lock);<BR>+ fastlock_destroy(&rs->cq_lock);<BR>+ fastlock_destroy(&rs->rlock);<BR>+ fastlock_destroy(&rs->slock);<BR>+ free(rs);<BR>+}<BR>+<BR>+static
void rs_set_conn_data(struct rsocket *rs, struct rdma_conn_param
*param,<BR>+ struct rs_conn_data
*conn)<BR>+{<BR>+ conn->version = 1;<BR>+ conn->flags =
rs_host_is_net() ? RS_CONN_FLAG_NET : 0;<BR>+ conn->credits =
htons(rs->rq_size);<BR>+ conn->reserved2 =
0;<BR>+<BR>+ conn->target_sgl.addr = htonll((uintptr_t)
rs->target_sgl);<BR>+ conn->target_sgl.length =
htonl(RS_SGL_SIZE);<BR>+ conn->target_sgl.key =
htonl(rs->target_mr->rkey);<BR>+<BR>+ conn->data_buf.addr =
htonll((uintptr_t) rs->rbuf);<BR>+ conn->data_buf.length =
htonl(rs->rbuf_size >> 1);<BR>+ conn->data_buf.key =
htonl(rs->rmr->rkey);<BR>+<BR>+ param->private_data =
conn;<BR>+ param->private_data_len = sizeof *conn;<BR>+}<BR>+<BR>+static
void rs_save_conn_data(struct rsocket *rs, struct rs_conn_data
*conn)<BR>+{<BR>+ rs->remote_sgl.addr =
ntohll(conn->target_sgl.addr);<BR>+ rs->remote_sgl.length =
ntohl(conn->target_sgl.length);<BR>+ rs->remote_sgl.key =
ntohl(conn->target_sgl.key);<BR>+ rs->remote_sge = 1;<BR>+ if
((rs_host_is_net() && !(conn->flags & RS_CONN_FLAG_NET))
||<BR>+ (!rs_host_is_net() && (conn->flags
& RS_CONN_FLAG_NET)))<BR>+ rs->opts =
RS_OPT_SWAP_SGL;<BR>+<BR>+ rs->target_sgl[0].addr =
ntohll(conn->data_buf.addr);<BR>+ rs->target_sgl[0].length =
ntohl(conn->data_buf.length);<BR>+ rs->target_sgl[0].key =
ntohl(conn->data_buf.key);<BR>+<BR>+ rs->sseq_comp =
ntohs(conn->credits);<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int
rsocket(int domain, int type, int protocol)<BR>+{<BR>+ struct rsocket
*rs;<BR>+ int ret;<BR>+<BR>+ wsa_setlasterror(0);<BR>+ if
((domain != PF_INET && domain != PF_INET6)
||<BR>+ (type != SOCK_STREAM) || (protocol &&
protocol != IPPROTO_TCP))<BR>+ return
ERR(ENOTSUP);<BR>+<BR>+ rs_configure();<BR>+ rs =
rs_alloc(NULL);<BR>+ if (!rs)<BR>+ return
ERR(ENOMEM);<BR>+<BR>+ ret = rdma_create_id(NULL, &rs->cm_id, rs,
RDMA_PS_TCP);<BR>+ if (ret)<BR>+ goto err;<BR>+<BR>+ ret =
rs_insert(rs);<BR>+ if (ret < 0)<BR>+ goto
err;<BR>+<BR>+ rs->cm_id->route.addr.src_addr.sa_family =
(ADDRESS_FAMILY)domain;<BR>+ return
rs->index;<BR>+<BR>+err:<BR>+ rs_free(rs);<BR>+ return
ret;<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int rbind(int socket, const struct
sockaddr *addr, socklen_t addrlen)<BR>+{<BR>+ struct rsocket
*rs;<BR>+ int ret;<BR>+<BR>+ wsa_setlasterror(0);<BR>+ rs =
(struct rsocket *)idm_at(&idm, socket);<BR>+ ret =
rdma_bind_addr(rs->cm_id, (struct sockaddr *) addr);<BR>+ if
(!ret)<BR>+ {<BR>+ rs->state =
rs_bound;<BR>+ }<BR>+ else<BR>+ {<BR>+ wsa_setlasterror(errno);<BR>+ }<BR>+ <BR>+ return
ret;<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int rlisten(int socket, int
backlog)<BR>+{<BR>+ struct rsocket *rs;<BR>+ int
ret;<BR>+<BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+<BR>+ if (backlog > RS_MAX_BACKLOG ||
backlog == SOMAXCONN)<BR>+ backlog =
RS_MAX_BACKLOG;<BR>+<BR>+ ret = rdma_listen(rs->cm_id,
backlog);<BR>+ if (!ret)<BR>+ {<BR>+ rs->state =
rs_listening;<BR>+ }<BR>+ else<BR>+ {<BR>+ wsa_setlasterror(errno);<BR>+ }<BR>+ <BR>+ return
ret;<BR>+}<BR>+<BR>+static int rs_do_connect(struct rsocket
*rs)<BR>+{<BR>+ struct rdma_conn_param param;<BR>+ struct rs_conn_data
creq, *cresp;<BR>+ struct sockaddr_storage dst_addr;<BR>+ int to,
ret;<BR>+<BR>+ switch (rs->state) {<BR>+ case
rs_init:<BR>+ case rs_bound:<BR>+resolve_addr:<BR>+ to = 1000
<<
rs->retries++;<BR>+ RtlCopyMemory(<BR>+ &dst_addr,<BR>+ &rs->cm_id->route.addr.dst_addr,<BR>+ rs->cm_id->route.addr.dst_addr.sa_family
== AF_INET<BR>+ ? sizeof(struct
sockaddr_in)<BR>+ : sizeof(struct
sockaddr_in6)<BR>+ );<BR>+ ret =
rdma_resolve_addr(rs->cm_id, NULL,<BR>+ (struct
sockaddr *)&dst_addr, to);<BR>+ if
(!ret)<BR>+ goto resolve_route;<BR>+ if (errno ==
EAGAIN || errno == EWOULDBLOCK)<BR>+ rs->state =
rs_resolving_addr;<BR>+ break;<BR>+ case
rs_resolving_addr:<BR>+ ret =
ucma_complete(rs->cm_id);<BR>+ if (ret) {<BR>+ if
(errno == ETIMEDOUT && rs->retries <=
RS_CONN_RETRIES)<BR>+ goto
resolve_addr;<BR>+ break;<BR>+ }<BR>+<BR>+ rs->retries
= 0;<BR>+resolve_route:<BR>+ to = 1000 <<
rs->retries++;<BR>+ ret = rdma_resolve_route(rs->cm_id,
to);<BR>+ if (!ret)<BR>+ goto
do_connect;<BR>+ if (errno == EAGAIN || errno ==
EWOULDBLOCK)<BR>+ rs->state =
rs_resolving_route;<BR>+ break;<BR>+ case
rs_resolving_route:<BR>+ ret =
ucma_complete(rs->cm_id);<BR>+ if (ret) {<BR>+ if
(errno == ETIMEDOUT && rs->retries <=
RS_CONN_RETRIES)<BR>+ goto
resolve_route;<BR>+ break;<BR>+ }<BR>+do_connect:<BR>+ ret
= rs_create_ep(rs);<BR>+ if
(ret)<BR>+ break;<BR>+<BR>+ memset(¶m, 0,
sizeof param);<BR>+ rs_set_conn_data(rs, ¶m,
&creq);<BR>+ param.flow_control =
1;<BR>+ param.retry_count = 7;<BR>+ param.rnr_retry_count
= 7;<BR>+ rs->retries = 0;<BR>+<BR>+ ret =
rdma_connect(rs->cm_id, ¶m);<BR>+ if
(!ret)<BR>+ goto connected;<BR>+ if (errno == EAGAIN
|| errno == EWOULDBLOCK)<BR>+ rs->state =
rs_connecting;<BR>+ break;<BR>+ case
rs_connecting:<BR>+ ret =
ucma_complete(rs->cm_id);<BR>+ if
(ret)<BR>+ break;<BR>+connected:<BR>+ cresp =
(struct rs_conn_data *)
rs->cm_id->event->param.conn.private_data;<BR>+ if
(cresp->version != 1) {<BR>+ ret =
ERR(ENOTSUP);<BR>+ break;<BR>+ }<BR>+<BR>+ rs_save_conn_data(rs,
cresp);<BR>+ rs->state =
rs_connect_rdwr;<BR>+ break;<BR>+ case
rs_accepting:<BR>+ if (!(rs->fd_flags &
O_NONBLOCK))<BR>+ rs_set_nonblocking(rs,
0);<BR>+<BR>+ ret = ucma_complete(rs->cm_id);<BR>+ if
(ret)<BR>+ break;<BR>+<BR>+ rs->state =
rs_connect_rdwr;<BR>+ break;<BR>+ default:<BR>+ ret =
ERR(EINVAL);<BR>+ break;<BR>+ }<BR>+<BR>+ if (ret)
{<BR>+ if (errno == EAGAIN || errno == EWOULDBLOCK)
{<BR>+ errno = EINPROGRESS;<BR>+ } else
{<BR>+ rs->state =
rs_connect_error;<BR>+ rs->err =
errno;<BR>+ }<BR>+ wsa_setlasterror(errno);<BR>+ }<BR>+ return
ret;<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int rconnect(int socket, const
struct sockaddr *addr, socklen_t addrlen)<BR>+{<BR>+ struct rsocket
*rs;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm,
socket);<BR>+ memcpy(&rs->cm_id->route.addr.dst_addr, addr,
addrlen);<BR>+ return rs_do_connect(rs);<BR>+}<BR>+<BR>+static int
rs_post_write(struct rsocket *rs,<BR>+ struct ibv_sge *sgl,
int nsge,<BR>+ uint32_t imm_data, int
flags,<BR>+ uint64_t addr, uint32_t
rkey)<BR>+{<BR>+ struct ibv_send_wr wr, *bad;<BR>+<BR>+ wr.wr_id =
(uint64_t) imm_data;<BR>+ wr.next = NULL;<BR>+ wr.sg_list =
sgl;<BR>+ wr.num_sge = nsge;<BR>+ wr.opcode =
IBV_WR_RDMA_WRITE_WITH_IMM;<BR>+ wr.send_flags =
flags;<BR>+ wr.imm_data = htonl(imm_data);<BR>+ wr.wr.rdma.remote_addr
= addr;<BR>+ wr.wr.rdma.rkey = rkey;<BR>+<BR>+ return
rdma_seterrno(ibv_post_send(rs->cm_id->qp, &wr,
&bad));<BR>+}<BR>+<BR>+/*<BR>+ * Update target SGE before sending
data. Otherwise the remote side may<BR>+ * update the entry before we
do.<BR>+ */<BR>+static int rs_write_data(struct rsocket
*rs,<BR>+ struct ibv_sge *sgl, int
nsge,<BR>+ uint32_t length, int
flags)<BR>+{<BR>+ uint64_t addr;<BR>+ uint32_t
rkey;<BR>+<BR>+ rs->sseq_no++;<BR>+ rs->sqe_avail--;<BR>+ rs->sbuf_bytes_avail
-= length;<BR>+<BR>+ addr =
rs->target_sgl[rs->target_sge].addr;<BR>+ rkey =
rs->target_sgl[rs->target_sge].key;<BR>+<BR>+ rs->target_sgl[rs->target_sge].addr
+= length;<BR>+ rs->target_sgl[rs->target_sge].length -=
length;<BR>+<BR>+ if (!rs->target_sgl[rs->target_sge].length)
{<BR>+ if (++rs->target_sge ==
RS_SGL_SIZE)<BR>+ rs->target_sge =
0;<BR>+ }<BR>+<BR>+ return rs_post_write(rs, sgl, nsge,
rs_msg_set(RS_OP_DATA, length),<BR>+
flags, addr, rkey);<BR>+}<BR>+<BR>+static uint32_t rs_sbuf_left(struct rsocket
*rs)<BR>+{<BR>+ return (uint32_t) (((uint64_t) (uintptr_t)
&rs->sbuf[rs->sbuf_size]) -<BR>+
rs->ssgl[0].addr);<BR>+}<BR>+<BR>+static void rs_send_credits(struct rsocket
*rs)<BR>+{<BR>+ struct ibv_sge ibsge;<BR>+ struct rs_sge
sge;<BR>+<BR>+ rs->ctrl_avail--;<BR>+ rs->rseq_comp =
rs->rseq_no + (rs->rq_size >> 1);<BR>+ if
((uint32_t)rs->rbuf_bytes_avail >= (rs->rbuf_size >> 1))
{<BR>+ if (!(rs->opts & RS_OPT_SWAP_SGL))
{<BR>+ sge.addr = (uintptr_t)
&rs->rbuf[rs->rbuf_free_offset];<BR>+ sge.key =
rs->rmr->rkey;<BR>+ sge.length = rs->rbuf_size
>> 1;<BR>+ } else {<BR>+ sge.addr =
cl_ntoh64((uintptr_t)
&rs->rbuf[rs->rbuf_free_offset]);<BR>+ sge.key =
cl_ntoh32(rs->rmr->rkey);<BR>+ sge.length =
cl_ntoh32(rs->rbuf_size >>
1);<BR>+ }<BR>+<BR>+ ibsge.addr = (uintptr_t)
&sge;<BR>+ ibsge.lkey = 0;<BR>+ ibsge.length =
sizeof(sge);<BR>+<BR>+ rs_post_write(rs, &ibsge,
1,<BR>+ rs_msg_set(RS_OP_SGL,
rs->rseq_no +
rs->rq_size),<BR>+
IBV_SEND_INLINE,<BR>+
rs->remote_sgl.addr +<BR>+
rs->remote_sge * sizeof(struct
rs_sge),<BR>+
rs->remote_sgl.key);<BR>+<BR>+ rs->rbuf_bytes_avail -=
rs->rbuf_size >> 1;<BR>+ rs->rbuf_free_offset +=
rs->rbuf_size >> 1;<BR>+ if
((uint32_t)rs->rbuf_free_offset >=
rs->rbuf_size)<BR>+ rs->rbuf_free_offset =
0;<BR>+ if (++rs->remote_sge ==
rs->remote_sgl.length)<BR>+ rs->remote_sge =
0;<BR>+ } else {<BR>+ rs_post_write(rs, NULL,
0,<BR>+ rs_msg_set(RS_OP_SGL,
rs->rseq_no + rs->rq_size), 0, 0, 0);<BR>+ }<BR>+}<BR>+<BR>+static
int rs_give_credits(struct rsocket *rs)<BR>+{<BR>+ return
(((uint32_t)rs->rbuf_bytes_avail >= (rs->rbuf_size >> 1))
||<BR>+ ((short) ((short)
rs->rseq_no - (short) rs->rseq_comp) >= 0))
&&<BR>+ rs->ctrl_avail
&& (rs->state & rs_connected);<BR>+}<BR>+<BR>+static void
rs_update_credits(struct rsocket *rs)<BR>+{<BR>+ if
(rs_give_credits(rs))<BR>+ rs_send_credits(rs);<BR>+}<BR>+<BR>+static
int rs_poll_cq(struct rsocket *rs)<BR>+{<BR>+ struct ibv_wc
wc;<BR>+ uint32_t imm_data;<BR>+ int ret, rcnt =
0;<BR>+<BR>+ while ((ret = ibv_poll_cq(rs->cm_id->recv_cq, 1,
&wc)) > 0) {<BR>+ if (wc.wr_id == RS_RECV_WR_ID)
{<BR>+ if (wc.status !=
IBV_WC_SUCCESS)<BR>+ continue;<BR>+ rcnt++;<BR>+<BR>+ imm_data
= ntohl(wc.imm_data);<BR>+ switch (rs_msg_op(imm_data))
{<BR>+ case
RS_OP_SGL:<BR>+ rs->sseq_comp = (uint16_t)
rs_msg_data(imm_data);<BR>+ break;<BR>+ case
RS_OP_CTRL:<BR>+ if (rs_msg_data(imm_data) ==
RS_CTRL_DISCONNECT) {<BR>+ rs->state =
rs_disconnected;<BR>+ return
0;<BR>+ } else if (rs_msg_data(imm_data) ==
RS_CTRL_SHUTDOWN) {<BR>+ rs->state &=
~rs_connect_rd;<BR>+ }<BR>+ break;<BR>+ default:<BR>+ rs->rmsg[rs->rmsg_tail].op
=
rs_msg_op(imm_data);<BR>+ rs->rmsg[rs->rmsg_tail].data
= rs_msg_data(imm_data);<BR>+ if (++rs->rmsg_tail ==
rs->rq_size + 1)<BR>+ rs->rmsg_tail =
0;<BR>+ break;<BR>+ }<BR>+ }
else {<BR>+ switch (rs_msg_op((uint32_t) wc.wr_id))
{<BR>+ case
RS_OP_SGL:<BR>+ rs->ctrl_avail++;<BR>+ break;<BR>+ case
RS_OP_CTRL:<BR>+ rs->ctrl_avail++;<BR>+ if
(rs_msg_data((uint32_t) wc.wr_id) ==
RS_CTRL_DISCONNECT)<BR>+ rs->state =
rs_disconnected;<BR>+ break;<BR>+ default:<BR>+ rs->sqe_avail++;<BR>+ rs->sbuf_bytes_avail
+= rs_msg_data((uint32_t)
wc.wr_id);<BR>+ break;<BR>+ }<BR>+<BR>+ if
(wc.status != IBV_WC_SUCCESS && (rs->state & rs_connected))
{<BR>+ rs->state =
rs_error;<BR>+ rs->err =
EIO;<BR>+ }<BR>+ }<BR>+ }<BR>+<BR>+ if
(rs->state & rs_connected) {<BR>+ while (!ret &&
rcnt--)<BR>+ {<BR>+ ret =
rs_post_recv(rs);<BR>+ }<BR>+ <BR>+ if (ret)
{<BR>+ rs->state =
rs_error;<BR>+ rs->err =
errno;<BR>+ }<BR>+ }<BR>+ return ret;<BR>+}<BR>+<BR>+static
int rs_get_cq_event(struct rsocket *rs)<BR>+{<BR>+ struct ibv_cq
*cq;<BR>+ void *context;<BR>+ int ret;<BR>+<BR>+ if
(!rs->cq_armed)<BR>+ {<BR>+ return
0;<BR>+ }<BR>+ ret =
ibv_get_cq_event(rs->cm_id->recv_cq_channel, &cq,
&context);<BR>+ if (!ret)
{<BR>+ ibv_ack_cq_events(rs->cm_id->recv_cq,
1);<BR>+ rs->cq_armed = 0;<BR>+ } else if (errno != EAGAIN)
{<BR>+ rs->state = rs_error;<BR>+ }<BR>+ return
ret;<BR>+}<BR>+<BR>+/*<BR>+ * Although we serialize rsend and rrecv calls with
respect to themselves,<BR>+ * both calls may run simultaneously and need to poll
the CQ for completions.<BR>+ * We need to serialize access to the CQ, but rsend
and rrecv need to<BR>+ * allow each other to make forward progress.<BR>+ *<BR>+
* For example, rsend may need to wait for credits from the remote side,<BR>+ *
which could be stalled until the remote process calls rrecv. This
should<BR>+ * not block rrecv from receiving data from the remote side
however.<BR>+ *<BR>+ * We handle this by using two locks. The cq_lock
protects against polling<BR>+ * the CQ and processing completions. The
cq_wait_lock serializes access to<BR>+ * waiting on the CQ.<BR>+ */<BR>+static
int rs_process_cq(struct rsocket *rs, int nonblock, int (*test)(struct rsocket
*rs))<BR>+{<BR>+ int
ret;<BR>+<BR>+ fastlock_acquire(&rs->cq_lock);<BR>+ do
{<BR>+ rs_update_credits(rs);<BR>+ ret =
rs_poll_cq(rs);<BR>+ if (test(rs)) {<BR>+ ret =
0;<BR>+ break;<BR>+ } else if (ret)
{<BR>+ break;<BR>+ } else if (nonblock)
{<BR>+ ret = ERR(EWOULDBLOCK);<BR>+ } else if
(!rs->cq_armed)
{<BR>+ ibv_req_notify_cq(rs->cm_id->recv_cq,
0);<BR>+ rs->cq_armed = 1;<BR>+ } else
{<BR>+ rs_update_credits(rs);<BR>+ fastlock_acquire(&rs->cq_wait_lock);<BR>+ fastlock_release(&rs->cq_lock);<BR>+ ret
=
rs_get_cq_event(rs);<BR>+ fastlock_release(&rs->cq_wait_lock);<BR>+ fastlock_acquire(&rs->cq_lock);<BR>+ }<BR>+ }
while
(!ret);<BR>+<BR>+ rs_update_credits(rs);<BR>+ fastlock_release(&rs->cq_lock);<BR>+ return
ret;<BR>+}<BR>+<BR>+static int rs_get_comp(struct rsocket *rs, int nonblock, int
(*test)(struct rsocket *rs))<BR>+{<BR>+ struct timeval s,
e;<BR>+ uint32_t poll_time = 0;<BR>+ int ret;<BR>+ do
{<BR>+ ret = rs_process_cq(rs, 1, test);<BR>+ if (!ret ||
nonblock || errno != EWOULDBLOCK) {<BR>+ return
ret;<BR>+ }<BR>+<BR>+ if
(!poll_time)<BR>+ gettimeofday(&s,
NULL);<BR>+<BR>+ gettimeofday(&e,
NULL);<BR>+ poll_time = (e.tv_sec - s.tv_sec) * 1000000
+<BR>+ (e.tv_usec - s.tv_usec) +
1;<BR>+ } while (poll_time <= polling_time);<BR>+ ret =
rs_process_cq(rs, 0, test);<BR>+ return ret;<BR>+}<BR>+<BR>+static int
rs_nonblocking(struct rsocket *rs)<BR>+{<BR>+ return (rs->fd_flags &
O_NONBLOCK);<BR>+}<BR>+<BR>+static int rs_is_cq_armed(struct rsocket
*rs)<BR>+{<BR>+ return rs->cq_armed;<BR>+}<BR>+<BR>+static int
rs_poll_all(struct rsocket *rs)<BR>+{<BR>+ return 1;<BR>+}<BR>+<BR>+/*<BR>+
* We use hardware flow control to prevent over running the remote<BR>+ * receive
queue. However, data transfers still require space in<BR>+ * the remote
rmsg queue, or we risk losing notification that data<BR>+ * has been
transfered.<BR>+ *<BR>+ * Be careful with race conditions in the check
below. The target SGL<BR>+ * may be updated by a remote RDMA write.<BR>+
*/<BR>+static int rs_can_send(struct rsocket *rs)<BR>+{<BR>+ return
rs->sqe_avail && rs->sbuf_bytes_avail
&&<BR>+ (rs->sseq_no !=
rs->sseq_comp) &&<BR>+
(rs->target_sgl[rs->target_sge].length != 0);<BR>+}<BR>+<BR>+static int
rs_conn_can_send(struct rsocket *rs)<BR>+{<BR>+ return rs_can_send(rs) ||
!(rs->state & rs_connect_wr);<BR>+}<BR>+<BR>+static int
rs_conn_can_send_ctrl(struct rsocket *rs)<BR>+{<BR>+ return
rs->ctrl_avail || !(rs->state & rs_connected);<BR>+}<BR>+<BR>+static
int rs_have_rdata(struct rsocket *rs)<BR>+{<BR>+ return (rs->rmsg_head
!= rs->rmsg_tail);<BR>+}<BR>+<BR>+static int rs_conn_have_rdata(struct
rsocket *rs)<BR>+{<BR>+ return rs_have_rdata(rs) || !(rs->state &
rs_connect_rd);<BR>+}<BR>+<BR>+static int rs_conn_all_sends_done(struct rsocket
*rs)<BR>+{<BR>+ return ((rs->sqe_avail + rs->ctrl_avail) ==
rs->sq_size) ||<BR>+ !(rs->state
& rs_connected);<BR>+}<BR>+<BR>+static ssize_t rs_peek(struct rsocket *rs,
void *buf, size_t len)<BR>+{<BR>+ size_t left = len;<BR>+ uint32_t
end_size, rsize;<BR>+ int rmsg_head, rbuf_offset;<BR>+ uint8_t *bufb =
(uint8_t *)buf;<BR>+<BR>+ rmsg_head =
rs->rmsg_head;<BR>+ rbuf_offset = rs->rbuf_offset;<BR>+<BR>+ for
(; left && (rmsg_head != rs->rmsg_tail); left -= rsize)
{<BR>+ if (left < rs->rmsg[rmsg_head].data)
{<BR>+ rsize = left;<BR>+ } else
{<BR>+ rsize =
rs->rmsg[rmsg_head].data;<BR>+ if (++rmsg_head ==
rs->rq_size + 1)<BR>+ rmsg_head =
0;<BR>+ }<BR>+<BR>+ end_size = rs->rbuf_size -
rbuf_offset;<BR>+ if (rsize > end_size)
{<BR>+ memcpy(bufb, &rs->rbuf[rbuf_offset],
end_size);<BR>+ rbuf_offset = 0;<BR>+ bufb +=
end_size;<BR>+ rsize -= end_size;<BR>+ left -=
end_size;<BR>+ }<BR>+ memcpy(bufb,
&rs->rbuf[rbuf_offset], rsize);<BR>+ rbuf_offset +=
rsize;<BR>+ bufb += rsize;<BR>+ }<BR>+<BR>+ return len -
left;<BR>+}<BR>+<BR>+/*<BR>+ * Continue to receive any queued data even if the
remote side has disconnected.<BR>+ */<BR>+ __declspec(dllexport)<BR>+ssize_t
rrecv(int socket, void *buf, size_t len, int flags)<BR>+{<BR>+ struct
rsocket *rs;<BR>+ size_t left = len;<BR>+ uint32_t end_size,
rsize;<BR>+ int ret;<BR>+ uint8_t *bufb = (uint8_t
*)buf;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+ if (rs->state & rs_opening)
{<BR>+ ret = rs_do_connect(rs);<BR>+ if (ret)
{<BR>+ if (errno ==
EINPROGRESS)<BR>+ errno =
EAGAIN;<BR>+ goto
out;<BR>+ }<BR>+ }<BR>+ fastlock_acquire(&rs->rlock);<BR>+ do
{<BR>+ if (!rs_have_rdata(rs)) {<BR>+ ret =
rs_get_comp(rs,
rs_nonblocking(rs),<BR>+ rs_conn_have_rdata);<BR>+ if
(ret)<BR>+ break;<BR>+ }<BR>+<BR>+ ret
= 0;<BR>+ if (flags & MSG_PEEK) {<BR>+ left =
len - rs_peek(rs, buf,
left);<BR>+ break;<BR>+ }<BR>+<BR>+ for
(; left && rs_have_rdata(rs); left -= rsize) {<BR>+ if
(left < rs->rmsg[rs->rmsg_head].data)
{<BR>+ rsize =
left;<BR>+ rs->rmsg[rs->rmsg_head].data -=
left;<BR>+ } else
{<BR>+ rs->rseq_no++;<BR>+ rsize
= rs->rmsg[rs->rmsg_head].data;<BR>+ if
(++rs->rmsg_head == rs->rq_size +
1)<BR>+ rs->rmsg_head =
0;<BR>+ }<BR>+<BR>+ end_size =
rs->rbuf_size - rs->rbuf_offset;<BR>+ if (rsize >
end_size) {<BR>+ memcpy(bufb,
&rs->rbuf[rs->rbuf_offset],
end_size);<BR>+ rs->rbuf_offset =
0;<BR>+ bufb +=
end_size;<BR>+ rsize -=
end_size;<BR>+ left -=
end_size;<BR>+ rs->rbuf_bytes_avail +=
end_size;<BR>+ }<BR>+ memcpy(bufb,
&rs->rbuf[rs->rbuf_offset],
rsize);<BR>+ rs->rbuf_offset +=
rsize;<BR>+ bufb +=
rsize;<BR>+ rs->rbuf_bytes_avail +=
rsize;<BR>+ }<BR>+ } while (left && (flags &
MSG_WAITALL) && (rs->state &
rs_connect_rd));<BR>+ <BR>+ fastlock_release(&rs->rlock);<BR>+<BR>+out:<BR>+ if
(ret)<BR>+ {<BR>+ wsa_setlasterror(errno);<BR>+ return
ret;<BR>+ }<BR>+ else<BR>+ {<BR>+ return len -
left;<BR>+ }<BR>+}<BR>+<BR>+ssize_t rrecvfrom(int socket, void *buf, size_t
len, int flags,<BR>+ struct sockaddr *src_addr, socklen_t
*addrlen)<BR>+{<BR>+ ssize_t ret;<BR>+<BR>+ ret = rrecv(socket, buf,
len, flags);<BR>+ if (ret > 0 &&
src_addr)<BR>+ rgetpeername(socket, src_addr,
addrlen);<BR>+<BR>+ return ret;<BR>+}<BR>+<BR>+/*<BR>+ * Simple,
straightforward implementation for now that only tries to fill<BR>+ * in the
first vector.<BR>+ */<BR>+static ssize_t rrecvv(int socket, const struct iovec
*iov, int iovcnt, int flags)<BR>+{<BR>+ return rrecv(socket,
iov[0].iov_base, iov[0].iov_len, flags);<BR>+}<BR>+<BR>+ssize_t rrecvmsg(int
socket, struct msghdr *msg, int flags)<BR>+{<BR>+ if (msg->msg_control
&& msg->msg_controllen)<BR>+ return
ERR(ENOTSUP);<BR>+<BR>+ return rrecvv(socket, msg->msg_iov, (int)
msg->msg_iovlen, msg->msg_flags);<BR>+}<BR>+<BR>+ssize_t rread(int socket,
void *buf, size_t count)<BR>+{<BR>+ return rrecv(socket, buf, count,
0);<BR>+}<BR>+<BR>+ssize_t rreadv(int socket, const struct iovec *iov, int
iovcnt)<BR>+{<BR>+ return rrecvv(socket, iov, iovcnt,
0);<BR>+}<BR>+<BR>+/*<BR>+ * We overlap sending the data, by posting a small
work request immediately,<BR>+ * then increasing the size of the send on each
iteration.<BR>+ */<BR>+ssize_t rsend(int socket, const void *buf, size_t len,
int flags)<BR>+{<BR>+ struct rsocket *rs;<BR>+ struct ibv_sge
sge;<BR>+ size_t left = len;<BR>+ uint32_t xfer_size, olen =
RS_OLAP_START_SIZE;<BR>+ int ret = 0;<BR>+ uint8_t *bufb = (uint8_t
*)buf;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+ if (rs->state & rs_opening)
{<BR>+ ret = rs_do_connect(rs);<BR>+ if (ret)
{<BR>+ if (errno ==
EINPROGRESS)<BR>+ errno =
EAGAIN;<BR>+ goto
out;<BR>+ }<BR>+ }<BR>+<BR>+ fastlock_acquire(&rs->slock);<BR>+ for
(; left; left -= xfer_size, bufb += xfer_size) {<BR>+ if
(!rs_can_send(rs)) {<BR>+ ret = rs_get_comp(rs,
rs_nonblocking(rs),<BR>+
rs_conn_can_send);<BR>+ if
(ret)<BR>+ break;<BR>+ if
(!(rs->state & rs_connect_wr)) {<BR>+ ret =
ERR(ECONNRESET);<BR>+ break;<BR>+ }<BR>+ }<BR>+<BR>+ if
(olen < left) {<BR>+ xfer_size =
olen;<BR>+ if (olen <
RS_MAX_TRANSFER)<BR>+ olen <<= 1;<BR>+ }
else {<BR>+ xfer_size =
left;<BR>+ }<BR>+<BR>+ if (xfer_size >
(uint32_t)rs->sbuf_bytes_avail)<BR>+ xfer_size =
rs->sbuf_bytes_avail;<BR>+ if (xfer_size >
rs->target_sgl[rs->target_sge].length)<BR>+ xfer_size =
rs->target_sgl[rs->target_sge].length;<BR>+<BR>+ if (xfer_size
<= rs->sq_inline) {<BR>+ sge.addr =
(uintptr_t)bufb;<BR>+ sge.length =
xfer_size;<BR>+ sge.lkey = 0;<BR>+ ret =
rs_write_data(rs, &sge, 1, xfer_size, IBV_SEND_INLINE);<BR>+ }
else if (xfer_size <= rs_sbuf_left(rs)) {<BR>+ memcpy((void
*) (uintptr_t) rs->ssgl[0].addr, bufb,
xfer_size);<BR>+ rs->ssgl[0].length =
xfer_size;<BR>+ ret = rs_write_data(rs, rs->ssgl, 1,
xfer_size, 0);<BR>+ if (xfer_size <
rs_sbuf_left(rs))<BR>+ rs->ssgl[0].addr +=
xfer_size;<BR>+ else<BR>+ rs->ssgl[0].addr
= (uintptr_t) rs->sbuf;<BR>+ } else
{<BR>+ rs->ssgl[0].length =
rs_sbuf_left(rs);<BR>+ memcpy((void *) (uintptr_t)
rs->ssgl[0].addr,
bufb,<BR>+ rs->ssgl[0].length);<BR>+ rs->ssgl[1].length
= xfer_size - rs->ssgl[0].length;<BR>+ memcpy(rs->sbuf,
bufb + rs->ssgl[0].length, rs->ssgl[1].length);<BR>+ ret
= rs_write_data(rs, rs->ssgl, 2, xfer_size,
0);<BR>+ rs->ssgl[0].addr = (uintptr_t) rs->sbuf +
rs->ssgl[1].length;<BR>+ }<BR>+ if
(ret)<BR>+ break;<BR>+ }<BR>+ fastlock_release(&rs->slock);<BR>+<BR>+out:<BR>+ if
(ret && left ==
len)<BR>+ {<BR>+ wsa_setlasterror(errno);<BR>+ return
ret;<BR>+ }<BR>+ else<BR>+ {<BR>+ return len -
left;<BR>+ }<BR>+}<BR>+<BR>+ssize_t rsendto(int socket, const void *buf,
size_t len, int flags,<BR>+ const struct sockaddr *dest_addr,
socklen_t addrlen)<BR>+{<BR>+/*<BR>+ * In Windows on a connection-oriented
socket,<BR>+ * the dest_addr and addrlen parameters are just ignored,<BR>+ *
making sendto() equivalent to send().<BR>+ */<BR>+ return rsend(socket,
buf, len, flags);<BR>+}<BR>+<BR>+static void rs_copy_iov(void *dst, const struct
iovec **iov, size_t *offset, size_t len)<BR>+{<BR>+ size_t
size;<BR>+ uint8_t *dstb = (uint8_t *)dst;<BR>+<BR>+ while (len)
{<BR>+ size = (*iov)->iov_len - *offset;<BR>+ if (size
> len) {<BR>+ memcpy (dstb, (*iov)->iov_base + *offset,
len);<BR>+ *offset +=
len;<BR>+ break;<BR>+ }<BR>+<BR>+ memcpy(dstb,
(*iov)->iov_base + *offset, size);<BR>+ len -=
size;<BR>+ dstb +=
size;<BR>+ (*iov)++;<BR>+ *offset =
0;<BR>+ }<BR>+}<BR>+<BR>+static ssize_t rsendv(int socket, const struct
iovec *iov, int iovcnt, int flags)<BR>+{<BR>+ struct rsocket
*rs;<BR>+ const struct iovec *cur_iov;<BR>+ size_t left, len, offset =
0;<BR>+ uint32_t xfer_size, olen = RS_OLAP_START_SIZE;<BR>+ int i, ret
= 0;<BR>+<BR>+ rs = (struct rsocket *)idm_at(&idm,
socket);<BR>+ if (rs->state & rs_opening) {<BR>+ ret =
rs_do_connect(rs);<BR>+ if (ret) {<BR>+ if (errno ==
EINPROGRESS)<BR>+ errno =
EAGAIN;<BR>+ return
ret;<BR>+ }<BR>+ }<BR>+<BR>+ cur_iov = iov;<BR>+ len =
iov[0].iov_len;<BR>+ for (i = 1; i < iovcnt; i++)<BR>+ len +=
iov[i].iov_len;<BR>+<BR>+ fastlock_acquire(&rs->slock);<BR>+ for
(left = len; left; left -= xfer_size) {<BR>+ if (!rs_can_send(rs))
{<BR>+ ret = rs_get_comp(rs,
rs_nonblocking(rs),<BR>+
rs_conn_can_send);<BR>+ if
(ret)<BR>+ break;<BR>+ if
(!(rs->state & rs_connect_wr)) {<BR>+ ret =
ERR(ECONNRESET);<BR>+ break;<BR>+ }<BR>+ }<BR>+<BR>+ if
(olen < left) {<BR>+ xfer_size =
olen;<BR>+ if (olen <
RS_MAX_TRANSFER)<BR>+ olen <<= 1;<BR>+ }
else {<BR>+ xfer_size =
left;<BR>+ }<BR>+<BR>+ if (xfer_size >
(uint32_t)rs->sbuf_bytes_avail)<BR>+ xfer_size =
rs->sbuf_bytes_avail;<BR>+ if (xfer_size >
rs->target_sgl[rs->target_sge].length)<BR>+ xfer_size =
rs->target_sgl[rs->target_sge].length;<BR>+<BR>+ if (xfer_size
<= rs_sbuf_left(rs)) {<BR>+ rs_copy_iov((void *) (uintptr_t)
rs->ssgl[0].addr,<BR>+
&cur_iov, &offset,
xfer_size);<BR>+ rs->ssgl[0].length =
xfer_size;<BR>+ ret = rs_write_data(rs, rs->ssgl, 1,
xfer_size,<BR>+ xfer_size <=
rs->sq_inline ? IBV_SEND_INLINE : 0);<BR>+ if (xfer_size
< rs_sbuf_left(rs))<BR>+ rs->ssgl[0].addr +=
xfer_size;<BR>+ else<BR>+ rs->ssgl[0].addr
= (uintptr_t) rs->sbuf;<BR>+ } else
{<BR>+ rs->ssgl[0].length =
rs_sbuf_left(rs);<BR>+ rs_copy_iov((void *) (uintptr_t)
rs->ssgl[0].addr,
&cur_iov,<BR>+ &offset,
rs->ssgl[0].length);<BR>+ rs->ssgl[1].length = xfer_size
- rs->ssgl[0].length;<BR>+ rs_copy_iov(rs->sbuf,
&cur_iov, &offset, rs->ssgl[1].length);<BR>+ ret =
rs_write_data(rs, rs->ssgl, 2,
xfer_size,<BR>+ xfer_size <=
rs->sq_inline ? IBV_SEND_INLINE :
0);<BR>+ rs->ssgl[0].addr = (uintptr_t) rs->sbuf +
rs->ssgl[1].length;<BR>+ }<BR>+ if
(ret)<BR>+ break;<BR>+ }<BR>+ fastlock_release(&rs->slock);<BR>+<BR>+ return
(ret && left == len) ? ret : len -
left;<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+ssize_t rsendmsg(int socket,
const struct msghdr *msg, int
flags)<BR>+{<BR>+ wsa_setlasterror(0);<BR>+ if (msg->msg_control
&& msg->msg_controllen)<BR>+ return
ERR(ENOTSUP);<BR>+<BR>+ return rsendv(socket, msg->msg_iov, (int)
msg->msg_iovlen,
msg->msg_flags);<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+ssize_t rwrite(int
socket, const void *buf, size_t count)<BR>+{<BR>+ return rsend(socket, buf,
count, 0);<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+ssize_t rwritev(int socket,
const struct iovec *iov, int iovcnt)<BR>+{<BR>+ return rsendv(socket, iov,
iovcnt, 0);<BR>+}<BR>+<BR>+static struct pollfd *rs_fds_alloc(nfds_t
nfds)<BR>+{<BR>+ static __thread struct pollfd *rfds;<BR>+ static
__thread nfds_t rnfds;<BR>+<BR>+ if (nfds > rnfds) {<BR>+ if
(rfds)<BR>+ free(rfds);<BR>+<BR>+ rfds = (struct
pollfd *)malloc(sizeof *rfds * nfds);<BR>+ rnfds = rfds ? nfds :
0;<BR>+ }<BR>+<BR>+ return rfds;<BR>+}<BR>+<BR>+static int
rs_poll_rs(struct rsocket *rs, int
events,<BR>+ int nonblock, int
(*test)(struct rsocket *rs))<BR>+{<BR>+ COMP_SET fds;<BR>+ short
revents;<BR>+ int ret;<BR>+<BR>+check_cq:<BR>+ if ((rs->state &
rs_connected) || (rs->state == rs_disconnected)
||<BR>+ (rs->state & rs_error))
{<BR>+ rs_process_cq(rs, nonblock,
test);<BR>+<BR>+ revents = 0;<BR>+ if ((events &
POLLIN) && rs_conn_have_rdata(rs))<BR>+ revents |=
POLLIN;<BR>+ if ((events & POLLOUT) &&
rs_can_send(rs))<BR>+ revents |= POLLOUT;<BR>+ if
(!(rs->state & rs_connected)) {<BR>+ if (rs->state ==
rs_disconnected)<BR>+ revents |=
POLLHUP;<BR>+ else<BR>+ revents |=
POLLERR;<BR>+ }<BR>+<BR>+ return
revents;<BR>+ }<BR>+<BR>+ if (rs->state == rs_listening)
{<BR>+ if (CompSetInit(&fds))<BR>+ return
0;<BR>+<BR>+ CompSetZero(&fds);<BR>+ CompSetAdd(&rs->cm_id->channel->channel,
&fds);<BR>+ return CompSetPoll(&fds,
0);<BR>+ }<BR>+<BR>+ if (rs->state & rs_opening)
{<BR>+ ret = rs_do_connect(rs);<BR>+ if (ret)
{<BR>+ if (errno == EINPROGRESS)
{<BR>+ errno = 0;<BR>+ return
0;<BR>+ } else {<BR>+ return
POLLOUT;<BR>+ }<BR>+ }<BR>+ goto
check_cq;<BR>+ }<BR>+<BR>+ if (rs->state ==
rs_connect_error)<BR>+ return (rs->err && events &
POLLOUT) ? POLLOUT : 0;<BR>+<BR>+ return 0;<BR>+}<BR>+<BR>+static int
rs_poll_check(struct pollfd *fds, nfds_t nfds)<BR>+{<BR>+ struct rsocket
*rs;<BR>+ int i, cnt = 0;<BR>+<BR>+ for (i = 0; i < (int)nfds; i++)
{<BR>+ rs = (struct rsocket *)idm_lookup(&idm,
(int)fds[i].fd);<BR>+ if (rs)<BR>+ fds[i].revents =
(SHORT)rs_poll_rs(rs, fds[i].events, 1,
rs_poll_all);<BR>+ else<BR>+ ;//WSAPoll(&fds[i],
1, 0);<BR>+<BR>+ if
(fds[i].revents)<BR>+ cnt++;<BR>+ }<BR>+ return
cnt;<BR>+}<BR>+<BR>+static int rs_poll_arm(struct pollfd *rfds, struct pollfd
*fds, nfds_t nfds)<BR>+{<BR>+ struct rsocket *rs;<BR>+ int
i;<BR>+<BR>+ for (i = 0; i < (int)nfds; i++) {<BR>+ rs =
(struct rsocket *)idm_lookup(&idm, (int)fds[i].fd);<BR>+ if (rs)
{<BR>+ fds[i].revents = (SHORT)rs_poll_rs(rs, fds[i].events, 0,
rs_is_cq_armed);<BR>+ if
(fds[i].revents)<BR>+ return
1;<BR>+<BR>+ if (rs->state >=
rs_connected)<BR>+ rfds[i].fd =
((short)rs->cm_id->recv_cq_channel->comp_channel.Event >>
2);<BR>+ else<BR>+ rfds[i].fd =
((short)rs->cm_id->channel->channel.Event >>
2);<BR>+<BR>+ rfds[i].events = POLLIN;<BR>+ } else
{<BR>+ rfds[i].fd =
fds[i].fd;<BR>+ rfds[i].events =
fds[i].events;<BR>+ }<BR>+ rfds[i].revents =
0;<BR>+ }<BR>+ return 0;<BR>+}<BR>+<BR>+static int
rs_poll_events(struct pollfd *rfds, struct pollfd *fds, nfds_t
nfds)<BR>+{<BR>+ struct rsocket *rs;<BR>+ int i, cnt =
0;<BR>+<BR>+ for (i = 0; i < (int)nfds; i++) {<BR>+ if
(!rfds[i].revents)<BR>+ continue;<BR>+<BR>+ rs =
(struct rsocket *)idm_lookup(&idm, (int)fds[i].fd);<BR>+ if (rs)
{<BR>+ rs_get_cq_event(rs);<BR>+ fds[i].revents
= (SHORT)rs_poll_rs(rs, fds[i].events, 1, rs_poll_all);<BR>+ } else
{<BR>+ fds[i].revents =
rfds[i].revents;<BR>+ }<BR>+ if
(fds[i].revents)<BR>+ cnt++;<BR>+ }<BR>+ return
cnt;<BR>+}<BR>+<BR>+/*<BR>+ * We need to poll *all* fd's that the user specifies
at least once.<BR>+ * Note that we may receive events on an rsocket that may not
be reported<BR>+ * to the user (e.g. connection events or credit updates).
Process those<BR>+ * events, then return to polling until we find ones of
interest.<BR>+ */<BR>+__declspec(dllexport)<BR>+int rpoll(struct pollfd *fds,
nfds_t nfds, int timeout)<BR>+{<BR>+ struct timeval s, e;<BR>+ struct
pollfd *rfds;<BR>+ uint32_t poll_time = 0;<BR>+ int
ret;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ do
{<BR>+ ret = rs_poll_check(fds, nfds);<BR>+ if (ret ||
!timeout)<BR>+ return ret;<BR>+<BR>+ if
(!poll_time)<BR>+ gettimeofday(&s,
NULL);<BR>+<BR>+ gettimeofday(&e,
NULL);<BR>+ poll_time = (e.tv_sec - s.tv_sec) * 1000000
+<BR>+ (e.tv_usec - s.tv_usec) +
1;<BR>+ } while (poll_time <= polling_time);<BR>+<BR>+ rfds =
rs_fds_alloc(nfds);<BR>+ if (!rfds)<BR>+ return
ERR(ENOMEM);<BR>+<BR>+ do {<BR>+ ret = rs_poll_arm(rfds, fds,
nfds);<BR>+ if (ret)<BR>+ break;<BR>+#if
0<BR>+ ret = WSAPoll(rfds, nfds, timeout);<BR>+ if (ret
<= 0)<BR>+ break;<BR>+#endif<BR>+ ret =
rs_poll_events(rfds, fds, nfds);<BR>+ } while (!ret);<BR>+<BR>+ return
ret;<BR>+}<BR>+<BR>+static struct pollfd *<BR>+rs_select_to_poll(int *nfds,
fd_set *readfds, fd_set *writefds, fd_set *exceptfds)<BR>+{<BR>+ struct
pollfd *fds;<BR>+ int fd, i = 0;<BR>+<BR>+ fds = (struct pollfd
*)calloc(*nfds, sizeof *fds);<BR>+ if (!fds)<BR>+ return
NULL;<BR>+<BR>+ for (fd = 0; fd < *nfds; fd++) {<BR>+ if
(readfds && FD_ISSET(fd, readfds)) {<BR>+ fds[i].fd =
fd;<BR>+ fds[i].events =
POLLIN;<BR>+ }<BR>+<BR>+ if (writefds &&
FD_ISSET(fd, writefds)) {<BR>+ fds[i].fd =
fd;<BR>+ fds[i].events |=
POLLOUT;<BR>+ }<BR>+<BR>+ if (exceptfds &&
FD_ISSET(fd, exceptfds))<BR>+ fds[i].fd =
fd;<BR>+<BR>+ if
(fds[i].fd)<BR>+ i++;<BR>+ }<BR>+<BR>+ *nfds =
i;<BR>+ return fds;<BR>+}<BR>+<BR>+static int<BR>+rs_poll_to_select(int
nfds, struct pollfd *fds, fd_set *readfds,<BR>+ fd_set
*writefds, fd_set *exceptfds)<BR>+{<BR>+ int i, cnt = 0;<BR>+<BR>+ for
(i = 0; i < nfds; i++) {<BR>+ if (readfds &&
(fds[i].revents & (POLLIN | POLLHUP)))
{<BR>+ FD_SET(fds[i].fd,
readfds);<BR>+ cnt++;<BR>+ }<BR>+<BR>+ if
(writefds && (fds[i].revents & POLLOUT))
{<BR>+ FD_SET(fds[i].fd,
writefds);<BR>+ cnt++;<BR>+ }<BR>+<BR>+ if
(exceptfds && (fds[i].revents & ~(POLLIN | POLLOUT)))
{<BR>+ FD_SET(fds[i].fd,
exceptfds);<BR>+ cnt++;<BR>+ }<BR>+ }<BR>+ return
cnt;<BR>+}<BR>+<BR>+static int rs_convert_timeout(struct timeval
*timeout)<BR>+{<BR>+ return !timeout ? -1
:<BR>+ timeout->tv_sec * 1000 + timeout->tv_usec /
1000;<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int rselect(int nfds, fd_set
*readfds, fd_set *writefds,<BR>+ fd_set *exceptfds,
struct timeval *timeout)<BR>+{<BR>+ struct pollfd *fds;<BR>+ int
ret;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ fds =
rs_select_to_poll(&nfds, readfds, writefds, exceptfds);<BR>+ if
(!fds)<BR>+ return ERR(ENOMEM);<BR>+<BR>+ ret = rpoll(fds, nfds,
rs_convert_timeout(timeout));<BR>+<BR>+ if
(readfds)<BR>+ FD_ZERO(readfds);<BR>+ if
(writefds)<BR>+ FD_ZERO(writefds);<BR>+ if
(exceptfds)<BR>+ FD_ZERO(exceptfds);<BR>+<BR>+ if (ret >
0)<BR>+ ret = rs_poll_to_select(nfds, fds, readfds, writefds,
exceptfds);<BR>+<BR>+ free(fds);<BR>+ return
ret;<BR>+}<BR>+<BR>+/*<BR>+ * For graceful disconnect, notify the remote side
that we're<BR>+ * disconnecting and wait until all outstanding sends
complete.<BR>+ */<BR>+__declspec(dllexport)<BR>+int rshutdown(int socket, int
how)<BR>+{<BR>+ struct rsocket *rs;<BR>+ int ctrl, ret =
0;<BR>+<BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+ if (how == SHUT_RD)
{<BR>+ rs->state &= ~rs_connect_rd;<BR>+ return
0;<BR>+ }<BR>+<BR>+ if (rs->fd_flags &
O_NONBLOCK)<BR>+ rs_set_nonblocking(rs, 0);<BR>+<BR>+ if
(rs->state & rs_connected) {<BR>+ if (how == SHUT_RDWR)
{<BR>+ ctrl =
RS_CTRL_DISCONNECT;<BR>+ rs->state &= ~(rs_connect_rd |
rs_connect_wr);<BR>+ } else {<BR>+ rs->state
&= ~rs_connect_wr;<BR>+ ctrl = (rs->state &
rs_connect_rd) ?<BR>+ RS_CTRL_SHUTDOWN :
RS_CTRL_DISCONNECT;<BR>+ }<BR>+ if (!rs->ctrl_avail)
{<BR>+ ret = rs_process_cq(rs, 0,
rs_conn_can_send_ctrl);<BR>+ if
(ret)<BR>+ {<BR>+ wsa_setlasterror(errno);<BR>+ return
ret;<BR>+ }<BR>+ }<BR>+ if ((rs->state
& rs_connected) && rs->ctrl_avail)
{<BR>+ rs->ctrl_avail--;<BR>+ ret =
rs_post_write(rs, NULL, 0,<BR>+
rs_msg_set(RS_OP_CTRL, ctrl), 0, 0,
0);<BR>+ }<BR>+ }<BR>+<BR>+ if (rs->state &
rs_connected)<BR>+ rs_process_cq(rs, 0,
rs_conn_all_sends_done);<BR>+<BR>+ if ((rs->fd_flags & O_NONBLOCK)
&& (rs->state &
rs_connected))<BR>+ rs_set_nonblocking(rs, 1);<BR>+<BR>+ return
0;<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int rclose(int
socket)<BR>+{<BR>+ struct rsocket
*rs;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+ if (rs->state &
rs_connected)<BR>+ rshutdown(socket,
SHUT_RDWR);<BR>+<BR>+ rs_free(rs);<BR>+ return
0;<BR>+}<BR>+<BR>+static void rs_copy_addr(struct sockaddr *dst, struct sockaddr
*src, socklen_t *len)<BR>+{<BR>+ socklen_t size;<BR>+<BR>+ if
(src->sa_family == AF_INET) {<BR>+ size = min(*len, sizeof(struct
sockaddr_in));<BR>+ *len = sizeof(struct sockaddr_in);<BR>+ }
else {<BR>+ size = min(*len, sizeof(struct
sockaddr_in6));<BR>+ *len = sizeof(struct
sockaddr_in6);<BR>+ }<BR>+ memcpy(dst, src,
size);<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int rgetpeername(int socket,
struct sockaddr *addr, socklen_t *addrlen)<BR>+{<BR>+ struct rsocket
*rs;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+ rs_copy_addr(addr,
rdma_get_peer_addr(rs->cm_id), addrlen);<BR>+ return
0;<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int rgetsockname(int socket, struct
sockaddr *addr, socklen_t *addrlen)<BR>+{<BR>+ struct rsocket
*rs;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+ rs_copy_addr(addr,
rdma_get_local_addr(rs->cm_id), addrlen);<BR>+ return
0;<BR>+}<BR>+<BR>+/*<BR>+ * Nonblocking is usually not inherited between
sockets, but we need to<BR>+ * inherit it here to establish the connection
only. This is needed to<BR>+ * prevent rdma_accept from blocking until the
remote side finishes<BR>+ * establishing the connection. If we were to
allow rdma_accept to block,<BR>+ * then a single thread cannot establish a
connection with itself, or<BR>+ * two threads which try to connect to each other
can deadlock trying to<BR>+ * form a connection.<BR>+ *<BR>+ * Data transfers on
the new socket remain blocking unless the user<BR>+ * specifies otherwise
through rfcntl.<BR>+ */<BR>+__declspec(dllexport)<BR>+int raccept(int socket,
struct sockaddr *addr, socklen_t *addrlen)<BR>+{<BR>+ struct rsocket *rs,
*new_rs;<BR>+ struct rdma_conn_param param;<BR>+ struct rs_conn_data
*creq, cresp;<BR>+ int
ret;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+ new_rs = rs_alloc(rs);<BR>+ if
(!new_rs)<BR>+ return ERR(ENOMEM);<BR>+<BR>+ ret =
rdma_get_request(rs->cm_id, &new_rs->cm_id);<BR>+ if
(ret)<BR>+ goto err;<BR>+<BR>+ ret =
rs_insert(new_rs);<BR>+ if (ret < 0)<BR>+ goto
err;<BR>+<BR>+ creq = (struct rs_conn_data *)
new_rs->cm_id->event->param.conn.private_data;<BR>+ if
(creq->version != 1) {<BR>+ ret =
ERR(ENOTSUP);<BR>+ goto err;<BR>+ }<BR>+<BR>+ if
(rs->fd_flags & O_NONBLOCK)<BR>+ rs_set_nonblocking(new_rs,
O_NONBLOCK);<BR>+<BR>+ ret = rs_create_ep(new_rs);<BR>+ if
(ret)<BR>+ goto err;<BR>+<BR>+ rs_save_conn_data(new_rs,
creq);<BR>+ param =
new_rs->cm_id->event->param.conn;<BR>+ rs_set_conn_data(new_rs,
¶m, &cresp);<BR>+ ret = rdma_accept(new_rs->cm_id,
¶m);<BR>+ if (!ret)<BR>+ new_rs->state =
rs_connect_rdwr;<BR>+ else if (errno == EAGAIN || errno ==
EWOULDBLOCK)<BR>+ new_rs->state =
rs_accepting;<BR>+ else<BR>+ goto err;<BR>+<BR>+ if (addr
&& addrlen)<BR>+ rgetpeername(new_rs->index, addr,
addrlen);<BR>+ return
new_rs->index;<BR>+<BR>+err:<BR>+ rs_free(new_rs);<BR>+ return
ret;<BR>+}<BR>+<BR>+/*<BR>+ * Socket provider's variant:<BR>+ */<BR>+SOCKET
WSPAPI WSPAccept(<BR>+ SOCKET
socket,<BR>+ struct
sockaddr *addr,<BR>+ LPINT
addrlen,<BR>+ LPCONDITIONPROC
lpfnCondition,<BR>+
DWORD_PTR
dwCallbackData,<BR>+
LPINT lpErrno<BR>+ )<BR>+{<BR>+ struct rsocket *rs,
*new_rs;<BR>+ struct rdma_conn_param param;<BR>+ struct rs_conn_data
*creq, cresp;<BR>+ SOCKET new_socket =
INVALID_SOCKET;<BR>+ DWORD_PTR rsock =
INVALID_SOCKET;<BR>+ int ret =
gMainUpCallTable.lpWPUQuerySocketHandleContext(<BR>+ socket,<BR>+ (PDWORD_PTR)&rsock, //
__out PDWORD_PTR
lpContext<BR>+ lpErrno<BR>+ );<BR>+ if
(SOCKET_ERROR == ret)<BR>+ return
INVALID_SOCKET;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct
rsocket *)idm_at(&idm, (int)rsock);<BR>+ new_rs =
rs_alloc(rs);<BR>+ if (!new_rs)<BR>+ return
ERR(ENOMEM);<BR>+<BR>+ ret = rdma_get_request(rs->cm_id,
&new_rs->cm_id);<BR>+ if (ret)<BR>+ goto
err;<BR>+<BR>+ ret = rs_insert(new_rs);<BR>+ if (ret <
0)<BR>+ goto err;<BR>+<BR>+ creq = (struct rs_conn_data *)
new_rs->cm_id->event->param.conn.private_data;<BR>+ if
(creq->version != 1) {<BR>+ ret =
ERR(ENOTSUP);<BR>+ goto err;<BR>+ }<BR>+<BR>+ if
(rs->fd_flags & O_NONBLOCK)<BR>+ rs_set_nonblocking(new_rs,
O_NONBLOCK);<BR>+<BR>+ ret = rs_create_ep(new_rs);<BR>+ if
(ret)<BR>+ goto err;<BR>+<BR>+ rs_save_conn_data(new_rs,
creq);<BR>+ param =
new_rs->cm_id->event->param.conn;<BR>+ rs_set_conn_data(new_rs,
¶m, &cresp);<BR>+<BR>+ if (addr &&
addrlen)<BR>+ rgetpeername(new_rs->index, addr,
addrlen);<BR>+<BR>+ if (lpfnCondition) {<BR>+ struct sockaddr
local_addr;<BR>+ socklen_t
local_addrlen = sizeof(local_addr);<BR>+ WSABUF
caller_id;<BR>+ WSABUF callee_id;<BR>+ WSABUF
callee_data;<BR>+<BR>+ /* Set the caller and callee data buffer
*/<BR>+ caller_id.buf = (char *)addr;<BR>+ caller_id.len =
sizeof(*addr);<BR>+<BR>+ rs_copy_addr(&local_addr,
rdma_get_local_addr(new_rs->cm_id),
&local_addrlen);<BR>+<BR>+ callee_id.buf = (char
*)&local_addr;<BR>+ callee_id.len =
local_addrlen;<BR>+ <BR>+ callee_data.buf =
NULL;<BR>+ callee_data.len =
0;<BR>+<BR>+ switch(lpfnCondition(&caller_id, NULL, NULL, NULL,
&callee_id, &callee_data, NULL, dwCallbackData))
{<BR>+ default:<BR>+ /* Should never happen
*/<BR>+ /* Fall through. */<BR>+ case
CF_REJECT:<BR>+ *lpErrno =
WSAECONNREFUSED;<BR>+ ret =
(int)INVALID_SOCKET;<BR>+ goto err;<BR>+ case
CF_DEFER:<BR>+ *lpErrno =
WSATRY_AGAIN;<BR>+ ret =
(int)INVALID_SOCKET;<BR>+ goto err;<BR>+ case
CF_ACCEPT:<BR>+ break;<BR>+ }<BR>+ }<BR>+ new_socket
=
gMainUpCallTable.lpWPUCreateSocketHandle(<BR>+ gProtocolInfo.dwCatalogEntryId,<BR>+ new_rs->index, //
__in DWORD_PTR
dwContext<BR>+ lpErrno<BR>+ );<BR>+ if
(INVALID_SOCKET == new_socket) {<BR>+ //??? *lpErrno =
WSATRY_AGAIN;<BR>+ goto err;<BR>+ }<BR>+ ret =
rdma_accept(new_rs->cm_id, ¶m);<BR>+ if
(!ret)<BR>+ new_rs->state = rs_connect_rdwr;<BR>+ else if
(errno == EAGAIN || errno == EWOULDBLOCK)<BR>+ new_rs->state =
rs_accepting;<BR>+ else<BR>+ goto err;<BR>+<BR>+ return
new_socket;<BR>+<BR>+err:<BR>+ rs_free(new_rs);<BR>+ if (new_socket !=
INVALID_SOCKET)<BR>+ gMainUpCallTable.lpWPUCloseSocketHandle(new_socket,
lpErrno);<BR>+<BR>+ return
INVALID_SOCKET;<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int rsetsockopt(int
socket, int level, int optname,<BR>+ const void *optval, socklen_t
optlen)<BR>+{<BR>+ struct rsocket *rs;<BR>+ int ret, opt_on =
0;<BR>+ uint64_t *opts = NULL;<BR>+ <BR>+ ret =
ERR(ENOTSUP);<BR>+ rs = (struct rsocket *)idm_at(&idm,
socket);<BR>+ switch (level) {<BR>+ case
SOL_SOCKET:<BR>+ opts = &rs->so_opts;<BR>+ switch
(optname) {<BR>+ case SO_REUSEADDR:<BR>+ ret =
rdma_set_option(rs->cm_id,
RDMA_OPTION_ID,<BR>+
RDMA_OPTION_ID_REUSEADDR,<BR>+
(void *) optval, optlen);<BR>+ if (ret && ((errno ==
ENOSYS) || ((rs->state != rs_init)
&&<BR>+ rs->cm_id->context
&&<BR>+
(rs->cm_id->verbs->device->transport_type ==
IBV_TRANSPORT_IB))))<BR>+ ret =
0;<BR>+ opt_on = *(int *)
optval;<BR>+ break;<BR>+ case
SO_RCVBUF:<BR>+ if
(!rs->rbuf)<BR>+ rs->rbuf_size = (*(uint32_t *)
optval) << 1;<BR>+ ret =
0;<BR>+ break;<BR>+ case
SO_SNDBUF:<BR>+ if
(!rs->sbuf)<BR>+ rs->sbuf_size = (*(uint32_t *)
optval) << 1;<BR>+ ret =
0;<BR>+ break;<BR>+ case
SO_LINGER:<BR>+ /* Invert value so default so_opt = 0 is on
*/<BR>+ opt_on = !((struct linger *)
optval)->l_onoff;<BR>+ ret =
0;<BR>+ break;<BR>+ case
SO_KEEPALIVE:<BR>+ opt_on = *(int *)
optval;<BR>+ ret =
0;<BR>+ break;<BR>+ case
SO_OOBINLINE:<BR>+ opt_on = *(int *)
optval;<BR>+ ret =
0;<BR>+ break;<BR>+ default:<BR>+ break;<BR>+ }<BR>+ break;<BR>+ case
IPPROTO_TCP:<BR>+ opts = &rs->tcp_opts;<BR>+ switch
(optname) {<BR>+ case TCP_NODELAY:<BR>+ opt_on =
*(int *) optval;<BR>+ ret =
0;<BR>+ break;<BR>+ case
TCP_MAXSEG:<BR>+ ret =
0;<BR>+ break;<BR>+ default:<BR>+ break;<BR>+ }<BR>+ break;<BR>+ case
IPPROTO_IPV6:<BR>+ opts =
&rs->ipv6_opts;<BR>+ switch (optname) {<BR>+ case
IPV6_V6ONLY:<BR>+ ret = rdma_set_option(rs->cm_id,
RDMA_OPTION_ID,<BR>+
RDMA_OPTION_ID_AFONLY,<BR>+
(void *) optval, optlen);<BR>+ opt_on = *(int *)
optval;<BR>+ break;<BR>+ default:<BR>+ break;<BR>+ }<BR>+ case
SOL_RDMA:<BR>+ if (rs->state >= rs_opening)
{<BR>+ ret =
ERR(EINVAL);<BR>+ break;<BR>+ }<BR>+<BR>+ switch
(optname) {<BR>+ case
RDMA_SQSIZE:<BR>+ rs->sq_size = min((*(uint32_t *) optval),
RS_QP_MAX_SIZE);<BR>+ ret =
0;<BR>+ break;<BR>+ case
RDMA_RQSIZE:<BR>+ rs->rq_size = min((*(uint32_t *) optval),
RS_QP_MAX_SIZE);<BR>+ ret =
0;<BR>+ break;<BR>+ case
RDMA_INLINE:<BR>+ rs->sq_inline = min(*(uint32_t *) optval,
RS_QP_MAX_SIZE);<BR>+ if (rs->sq_inline <
RS_MIN_INLINE)<BR>+ rs->sq_inline =
RS_MIN_INLINE;<BR>+ ret =
0;<BR>+ break;<BR>+ default:<BR>+ break;<BR>+ }<BR>+ break;<BR>+ default:<BR>+ break;<BR>+ }<BR>+<BR>+ if
(!ret && opts) {<BR>+ if (opt_on)<BR>+ *opts
|= (uint64_t)(1 <<
optname);<BR>+ else<BR>+ *opts &= ~(1 <<
optname);<BR>+ }<BR>+<BR>+ return
ret;<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int rgetsockopt(int socket, int
level, int optname,<BR>+ void *optval, socklen_t
*optlen)<BR>+{<BR>+ struct rsocket *rs;<BR>+ int ret =
0;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+ switch (level) {<BR>+ case
SOL_SOCKET:<BR>+ switch (optname) {<BR>+ case
SO_REUSEADDR:<BR>+ case SO_KEEPALIVE:<BR>+ case
SO_OOBINLINE:<BR>+ *((int *) optval) = !!(rs->so_opts &
(uint64_t)(1 << optname));<BR>+ *optlen =
sizeof(int);<BR>+ break;<BR>+ case
SO_RCVBUF:<BR>+ *((int *) optval) =
rs->rbuf_size;<BR>+ *optlen =
sizeof(int);<BR>+ break;<BR>+ case
SO_SNDBUF:<BR>+ *((int *) optval) =
rs->sbuf_size;<BR>+ *optlen =
sizeof(int);<BR>+ break;<BR>+ case
SO_LINGER:<BR>+ /* Value is inverted so default so_opt = 0 is
on */<BR>+ ((struct linger *) optval)->l_onoff
=<BR>+ !(rs->so_opts & (uint64_t)(1 <<
optname));<BR>+ ((struct linger *) optval)->l_linger =
0;<BR>+ *optlen = sizeof(struct
linger);<BR>+ break;<BR>+ case
SO_ERROR:<BR>+ *((int *) optval) =
rs->err;<BR>+ *optlen =
sizeof(int);<BR>+ rs->err =
0;<BR>+ break;<BR>+ default:<BR>+ ret
=
ENOTSUP;<BR>+ break;<BR>+ }<BR>+ break;<BR>+ case
IPPROTO_TCP:<BR>+ switch (optname) {<BR>+ case
TCP_NODELAY:<BR>+ *((int *) optval) = !!(rs->tcp_opts &
(uint64_t)(1 << optname));<BR>+ *optlen =
sizeof(int);<BR>+ break;<BR>+ case
TCP_MAXSEG:<BR>+ *((int *) optval) = (rs->cm_id &&
rs->cm_id->route.num_paths)
?<BR>+ 1 << (7 +
rs->cm_id->route.path_rec->mtu)
:<BR>+
2048;<BR>+ *optlen =
sizeof(int);<BR>+ break;<BR>+ default:<BR>+ ret
=
ENOTSUP;<BR>+ break;<BR>+ }<BR>+ break;<BR>+ case
IPPROTO_IPV6:<BR>+ switch (optname) {<BR>+ case
IPV6_V6ONLY:<BR>+ *((int *) optval) = !!(rs->ipv6_opts &
(uint64_t)(1 << optname));<BR>+ *optlen =
sizeof(int);<BR>+ break;<BR>+ default:<BR>+ ret
=
ENOTSUP;<BR>+ break;<BR>+ }<BR>+ break;<BR>+ case
SOL_RDMA:<BR>+ switch (optname) {<BR>+ case
RDMA_SQSIZE:<BR>+ *((int *) optval) =
rs->sq_size;<BR>+ *optlen =
sizeof(int);<BR>+ break;<BR>+ case
RDMA_RQSIZE:<BR>+ *((int *) optval) =
rs->rq_size;<BR>+ *optlen =
sizeof(int);<BR>+ break;<BR>+ case
RDMA_INLINE:<BR>+ *((int *) optval) =
rs->sq_inline;<BR>+ *optlen =
sizeof(int);<BR>+ break;<BR>+ default:<BR>+ ret
=
ENOTSUP;<BR>+ break;<BR>+ }<BR>+ break;<BR>+ default:<BR>+ ret
= ENOTSUP;<BR>+ break;<BR>+ }<BR>+<BR>+ return
ERR/*rdma_seterrno*/(ret);<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int
rfcntl(int socket, int cmd, ... /* arg */ )<BR>+{<BR>+ struct rsocket
*rs;<BR>+ va_list args;<BR>+ long param;<BR>+ int ret =
0;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+ va_start(args, cmd);<BR>+ switch
(cmd) {<BR>+ case F_GETFL:<BR>+ return (int)
rs->fd_flags;<BR>+ case F_SETFL:<BR>+ param = va_arg(args,
long);<BR>+ if (param & O_NONBLOCK)<BR>+ ret =
rs_set_nonblocking(rs, O_NONBLOCK);<BR>+<BR>+ if
(!ret)<BR>+ rs->fd_flags |=
param;<BR>+ break;<BR>+ default:<BR>+ ret =
ERR(ENOTSUP);<BR>+ }<BR>+ va_end(args);<BR>+ return
ret;<BR>+}<BR>+<BR>+__declspec(dllexport)<BR>+int rioctlsocket(int socket, long
cmd, u_long* argp)<BR>+{<BR>+ struct rsocket
*rs;<BR>+ <BR>+ wsa_setlasterror(0);<BR>+ rs = (struct rsocket
*)idm_at(&idm, socket);<BR>+ switch (cmd) {<BR>+ case
FIONBIO:<BR>+ if
(*argp)<BR>+ rs->fd_flags |=
O_NONBLOCK;<BR>+ return rs_set_nonblocking(rs, *argp ?
O_NONBLOCK : 0);<BR>+ case FIONREAD:<BR>+ case
SIOCATMARK:<BR>+ return
ERR(ENOTSUP);<BR>+ default:<BR>+ return
ERR(ENOTSUP);<BR>+ }<BR>+}<BR>+<BR>+/**<BR>+ *
\brief Get current RSockets status information for
the calling process<BR>+ * (by calling
librdmacm.dll directly).<BR>+ *<BR>+ * \param lppStatusBuffer Pointer to a
buffer with an array of RS_STATUS information entries<BR>+
* to be allocated and returned. The
caller is responsible for<BR>+
* deallocating that buffer via free()
when it is no longer needed.<BR>+ *<BR>+ *
\return The number of RS_STATUS entries contained
in the status buffer<BR>+ * returned by
*lppStatusBuffer.<BR>+ */<BR>+int rsGetStatus ( __out LPRS_STATUS
*lppStatusBuffer )<BR>+{<BR>+ DWORD i,
e;<BR>+ DWORD dwNumEntries = 0;<BR>+ struct
rsocket *rs;<BR>+ char *pst = "";<BR>+<BR>+ for
(i = 0; i < IDX_MAX_INDEX; i++)<BR>+ {<BR>+ if
(idm_lookup(&idm,
i))<BR>+ dwNumEntries++;<BR>+ }<BR>+ <BR>+ if
( 0 == dwNumEntries ||<BR>+ NULL ==
(*lppStatusBuffer = (LPRS_STATUS)malloc(dwNumEntries *
sizeof(**lppStatusBuffer)))<BR>+
)<BR>+ {<BR>+ return 0;<BR>+ }<BR>+ <BR>+ for
(<BR>+ i = 0, e = 0;<BR>+ i < IDX_MAX_INDEX
&& e <
dwNumEntries;<BR>+ i++<BR>+ )<BR>+ {<BR>+ if
(rs = (struct rsocket *)idm_lookup(&idm,
i))<BR>+ {<BR>+ lppStatusBuffer[e]->src_addr =
rs->cm_id->route.addr.src_addr;<BR>+ lppStatusBuffer[e]->dst_addr
= rs->cm_id->route.addr.dst_addr;<BR>+ switch
(rs->state)<BR>+ {<BR>+ case
rs_init: pst =
"INIT"; break;<BR>+ case
rs_bound: pst =
"BOUND"; break;<BR>+ case
rs_listening: pst =
"LISTENING"; break;<BR>+ case
rs_opening: pst =
"OPENING"; break;<BR>+ case
rs_resolving_addr: pst =
"RESOLVING_ADDR"; break;<BR>+ case rs_resolving_route:pst
= "RESOLVING_ROUTE";break;<BR>+ case
rs_connecting: pst =
"CONNECTING"; break;<BR>+ case
rs_accepting: pst =
"ACCEPTING"; break;<BR>+ case
rs_connected: pst =
"CONNECTED"; break;<BR>+ case
rs_connect_wr: pst =
"CONNECT_WR"; break;<BR>+ case
rs_connect_rd: pst =
"CONNECT_RD"; break;<BR>+ case
rs_connect_rdwr: pst =
"CONNECT_RDWR"; break;<BR>+ case
rs_connect_error: pst =
"CONNECT_ERROR"; break;<BR>+ case
rs_disconnected: pst =
"DISCONNECTED"; break;<BR>+ case
rs_error: pst =
"ERROR"; break;<BR>+ default: pst
=
"UNKNOWN";<BR>+ }<BR>+ strncpy(lppStatusBuffer[e]->state,
pst, sizeof(lppStatusBuffer[e]->state) -
1);<BR>+ lppStatusBuffer[e]->state[sizeof(lppStatusBuffer[e]->state)]
=
'\0';<BR>+ e++;<BR>+ }<BR>+ }<BR>+<BR>+ return
e;<BR>+}<BR>Index:
ulp/librdmacm/src/Sources<BR>===================================================================<BR>---
ulp/librdmacm/src/Sources (revision 3419)<BR>+++
ulp/librdmacm/src/Sources (working copy)<BR>@@ -12,10 +12,12
@@<BR> cma.rc \<BR> cma_main.cpp \<BR> cma.cpp \<BR>- addrinfo.cpp<BR>+ addrinfo.cpp \<BR>+ rsocket.cpp \<BR>+ indexer.cpp<BR> <BR> INCLUDES
=
..\include;..\..\..\inc;..\..\..\inc\user;..\..\libibverbs\include;\<BR>-
..\..\..\inc\user\linux;<BR>+
..\..\..\inc\user\linux<BR> <BR> USER_C_FLAGS = $(USER_C_FLAGS)
-DEXPORT_CMA_SYMBOLS<BR> <BR>@@ -26,4 +28,5
@@<BR> $(SDK_LIB_PATH)\iphlpapi.lib \<BR> $(TARGETPATH)\*\ibat.lib \<BR> $(TARGETPATH)\*\libibverbs.lib \<BR>- $(TARGETPATH)\*\winverbs.lib<BR>+ $(TARGETPATH)\*\winverbs.lib
\<BR>+ $(TARGETPATH)\*\complib.lib<BR></DIV></SPAN></FONT></DIV><br clear=all> This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law.
If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message.
Thank you for your co-operation.
</BODY></HTML>